AFRL-RH-WP-TR-2016-0060 


Depth  Acuity  Methodology  for  Electronic  3D  Displays: 

e James  (ej) 


Eric  L.  Heft,  John  Mclntire, 
Frederick  M.  Meyer,  and  Darrel  G.  Hopper 

USAF  AFMC  711  HPW/RHCV 
2255  H  Street 

Wright- Patterson  AFB  OH  45433 

July  2016 


INTERIM  REPORT 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


STINFO  COPY 

AIR  FORCE  RESEARCH  LABORATORY 
711  HUMAN  PERFORMANCE  WING 
AIRMAN  SYSTEMS  DIRECTORATE 
WRIGHT-PATTERSON  AIR  FORCE  BASE  OH  45433-7022 
AIR  FORCE  MATERIEL  COMMAND 
UNITED  STATES  AIR  FORCE 


NOTICE  AND  SIGNATURE  PAGE 

Using  Government  drawings,  specifications,  or  other  data  included  in  this  document  for  any  purpose  other 
than  Government  procurement  does  not  in  any  way  obligate  the  U.S.  Government.  The  fact  that  the 
Government  formulated  or  supplied  the  drawings,  specifications,  or  other  data  does  not  license  the  holder 
or  any  other  person  or  corporation;  or  convey  any  rights  or  permission  to  manufacture,  use,  or  sell  any 
patented  invention  that  may  relate  to  them. 


AFRL-RH-WP-TR-20 16-0060  HAS  BEEN  REVIEWED  AND  IS  APPROVED  FOR  PUBLICATION  IN 
ACCORDANCE  WITH  ASSIGNED  DISTRIBUTION  STATEMENT. 


//signed// 

DARREL  G.  HOPPER 
Program  Manager 
Battlespace  Visualization  Branch 


//signed// 

ROBERT  C.  MCKINLEY 

Chief,  Battlespace  Visualization  Branch 

Warfighter  Interface  Division 


//signed// 

WILLIAM  E.  RUSSELL 
Chief,  Warfighter  Interface  Division 
Airman  Systems  Directorate 
711  Human  Performance  Wing 


This  report  is  published  in  the  interest  of  scientific  and  technical  information  exchange,  and  its 
publication  does  not  constitute  the  Government’s  approval  or  disapproval  of  its  ideas  or  findings. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 

The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to 
comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YY)  2.  REPORT  TYPE 

24-07-2016  Interim 

3.  DATES  COVERED  (From  -  To) 

11  Oct  2012  -  30  Jun  2016 

4.  TITLE  AND  SUBTITLE 

Depth  Acuity  Methodology  for 

Electronic  3D  Displays:  eJames  (eJ) 

5a.  CONTRACT  NUMBER 

FA8650-08-D-6801-0050 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

Multiple 

6.  AUTHOR(S) 

Eric  L.  Heft,  John  Mclntire, 

Frederick  M.  Meyer,  and  Darrel  G.  Hopper 

5d.  PROJECT  NUMBER 

5329 

5e.  TASK  NUMBER 

11 

5f.  WORK  UNIT  NUMBER 

HOCK  (53291102) 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

USAF  AFMC  71 1  HPW/RHCV 

2255  H  Street 

Wright-Patterson  AFB  OH  45433 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Materiel  Command 

Air  Force  Research  Laboratory 

711  Human  Performance  Wing,  Human  Effectiveness  Directorate 

Warfighter  Interface  Division,  Battlespace  Visualization  Branch 

Wright-Patterson  Air  Force  Base  OH  45433-7022 

10.  SPONSORING/MONITORING  AGENCY 
ACRONYM(S) 

USAF  AFMC  711  HPW/RHCV 

11.  SPONSORING/MONITORING  AGENCY 
REPORT  NUMBER(S) 

AFRL-RH- WP-TR-20 16-0060 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT  DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution 
unlimited.  Previously  cleared:  88ABW-2013-2508,  88 ABW-20 14-0320  88ABW-2014-1130 _ 


13.  SUPPLEMENTARY  NOTES  88 ABW  Cleared  09/08/2016;  88 ABW-20 16-4404.  Report  contains  color.  Contains  Patentable 

Information  (patent  being  pursued  but  not  approved). _ 

14.  ABSTRACT  An  electronic  version  (eJ)  of  the  James  (J)  test  of  depth  acuity  is  created  and  evaluated  on  three  different  types  of  3D  electronic 
displays:  one  active-eyewear  Stereo  3D  (S3D)  and  two  non-eyewear  full  parallax  Field-of-Light  Display  (FoLD)  systems.  The  two  FoLD 
systems  are  the  Actuality  Integral  Slice  (AIS)  swept-screen  volumetric  Perspecta  and  the  Zebra  Integral  Ray  (ZIR)  hogel-based  ZScape  Motion 
Display  (ZMD).  The  James  test  is  designed  so  that  the  only  factor  examined  is  binocular  single  vision  (BSV),  aka  stereoscopic  parallax.  The 
James  Depth  Perception  Apparatus  (DP A)  comprises  two  vertical  white  1cm  wide  rods  (or  thin  strings)  viewed  at  a  reference  distance  (e.g.  6m) 
against  a  black  background  through  a  window  in  a  black  front.  The  eJ  computer  graphics  white  bar  stimuli  in  this  work  are  designed  to  visually 
emulate — within  limitations  of  the  S3D,  AIS,  and  ZIR — the  white  rods  in  the  James  DPA.  The  unique  artifacts  of  the  three  electronic  display 
types  impede  creation  of  a  single  graphical  representation  of  the  two-bar  stimuli  and  complicate  cross-type  comparisons.  A  compact  variant  of 
the  James  DPA  developed  by  Howard  made  the  left  rod  stationary  and  right  rod  adjustable  by  the  subject  (rather  than  the  administrator)  via  a 
hand-crank  to  match  distance  via  a  mechanical  string-pulley  system.  The  Howard  DPA  variant  also  enabled  depth  acuity  threshold  determination 
by  two  methods:  (a)  forced-choice  of  nearest  bar;  (b)  adjustment  of  right  bar  to  match  left.  An  AFRL-automated  Howard  DPA  is  included  in  the 
present  study  to  enable  a  comparison  of  real-world  stimuli  versus  the  electronic  displays.  A  hand-held  computer  game  controller  is  used  to 
automate  data  collection  and  subject  trial  initiation.  Four  software  interfaces  are  developed  to  control  the  stimuli  positions  presented  on  the  three 
3D  electronic  systems  and  the  automated  DPA;  the  right  bar  is  randomly  positioned  from  trial  to  trial.  The  four  systems  (AIS,  S3D,  ZIR,  DPA) 
are  set  up  side-by-side  for  a  shake-down  evaluation  using  six  subjects  tested  by  methods  (a)  and  (b).  The  subject  results  are  presented,  analyzed, 
and  discussed.  Binocular  depth  acuity  is  assessed  on  all  four  systems.  The  within-type  and  limited  cross-type  analyses  indicate  that  eJ  is  a  viable 
depth  acuity  metric  to  guide  and  assess  progress  in  the  development  of  the  emerging  FoLD  class  of  non-eyewear  3D  visualization  systems. 


15.  SUBJECT  TERMS  Depth  Acuity  Metric,  3D  Displays,  Stereo  3D,  S3D,  Volumetric  Display,  Holographic  Video  Display,  HVD,  Hogel 


16.  SECURITY  CLASSIFICATION  OF: 

17.LIMITATION 

18. 

19a.  NAME  OF  RESPONSIBLE  PERSON  (Monitor) 

NUMBER 

OF  ABSTRACT: 

a.  REPORT 

b.  ABSTRACT 

c.  THIS 

Darrel  G.  Hopper 

OF 

PAGE 

SAR 

PAGES 

19b.  TELEPHONE  NUMBER  (Include  Area  Code) 

Unclassified 

Unclassified 

Unclassified 

97 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39-18 


TABLE  OF  CONTENTS 

Section  Page 

List  of  Figures .  iii 

List  of  Tables .  v 

1.0  SUMMARY .  1 

2.0  INTRODUCTION .  4 

2.1  Metrology  Gaps  for  Emerging  Field-of-Light  Display  (FoFD)  Systems .  4 

2.2  FoFD  Systems  at  AFRF .  4 

2.3  AFRF  Programs  to  Develop  FoFD  Systems  and  Metrology .  5 

2.4  Depth  Acuity  Metric  for  the  Real  World  -  the  James  Test  and  Distance  Stereoscope  5 

2.5  Howard  Variation  of  the  James  Depth  Perception  Apparatus  (DPA):  Howard  DPA  1 1 

2.6  Devo  Modification  of  the  Howard  Six  Meter  Stereoscope:  The  Howard-Devo  DPA  12 

2.7  Speed  and  Accuracy  of  the  James  Test .  15 

2.8  Fimitations  of  the  James  Test  for  Understanding  Image  Perception  on  3D  Displays  15 

2.9  S3D  Systems  at  AFRF .  15 

2.10  Psychophysical  Research  on  Electronic  3D  Displays .  16 

3.0  METHODS,  ASSUMPTIONS  AND  PROCEDURES .  17 

3.1  Software  Suite  for  Graphical  Stimuli  Emulating  the  James  DPA  Sticks .  17 

3.2  Display  Class  &  Type:  Physical  Apparatus  &  Two-Vertical  Sticks .  18 

3.2.1  Device  Under  Test:  AFRF  Computer- Automated  Howard  DPA .  18 

3.2.2  Calibration  of  the  AFRF- Automated  Howard  DPA .  20 

3.3  Display  Class  &  Type:  S3D  &  Eye  Selection  Glasses  (ESG) .  21 

3.3.1  Rendering  for  Temporally  Multiplexed  Feft/Right  Eye  S3D  Perspective  Views .  21 

3.3.2  Device  Under  Test:  Dell  M6700  S3D  System  with  Fiquid  Crystal  Glasses .  21 

3.3.3  Calibration  of  S3D  Display .  22 

3.4  Display  Class  &  Type:  FoFD  &  Temporally  Scanned  Volume  (TSV) .  24 

3.4.1  Rendering  for  a  Rotating  Screen  TSV .  24 

3.4.2  Device  Under  Evaluation:  Actuality  Integral  Slice  (AIS)  Perspecta .  25 

3.4.3  Calibration  of  the  AIS  Perspecta .  25 

3.5  Display  Class  &  Type:  FoFD  &  Spatially  Integrated  Ray  (SIR) .  28 

3.5.1  Rendering  for  SIR  Display .  30 

3.5.2  Device  Under  Test:  Zebra  Integral  Ray  (ZIR)  1-Tile  Zscape  Motion  Display  (ZMD)  32 

3.5.3  Calibration  of  the  ZIR  1-Tile  ZMD .  32 

3.5.3. 1  Intra-Hogel  Alignment  of  the  ZIR  1-Tile  ZMD .  33 

3. 5. 3. 2  Display  Volume  Uniformity  of  the  ZIR  9-Tile  ZMD .  37 

3.6  Objective  Human  Subject  Evaluation  of  Proposed  eJ  Methodology .  40 

3.6.1  Experimental  Condition  for  eJ  Evaluation .  40 

3.6.2  Testing  Approaches .  40 

3.6.2. 1  Method  of  Adjustment  (Subject  Moves  Right  Rod  to  Match  Depth  of  Left) .  42 

3. 6. 2. 2  Method  of  Constant  Stimuli  (Forced  Choice:  Subject  Identifies  Nearer  Rod .  42 

3.6.3  Description  of  the  Subjects  Used  to  Evaluate  the  eJ  Methodology .  43 

3.6.3. 1  Subject  Selection  Criteria .  43 

3. 6. 3. 2  Anonymous  Demographic  Vectors .  43 

4.0  RESUFTS  AND  DISCUSSION .  44 

4.1  Implementation  of  the  Two-Bar  Stimuli  on  the  Three  Electronic  3D  Displays .  44 


l 


4.2  Two-Bar  Stimuli  Depth  Acuity  Testing  of  Six  Observers  on  the  Four  3D  Devices. . .  50 

4.2.1  Dataset  1:  Depth  Acuity  Threshold  Measured  by  Method  of  Adjustment  (MoA) .  50 

4.2. 1.1  MoA  Signed  Positional  Error  Data .  50 

4.2. 1.2  MoA  Absolute  Positional  Error  Data .  51 

4.2. 1.2.1  Mean  Placement  Performance  as  Function  of  Reference  Distance .  52 

4.2. 1 .2.2  Per  Trial  Placement  Performance  as  Function  of  Reference  Distance .  53 

4.2. 1.2.3  Stereoacuity  Threshold  Estimates  Based  on  MoA  Dataset  1 .  56 

4.2.2  Dataset  2:  Depth  Acuity  Threshold  Measured  by  Method  of  Constant  Stimuli  (MoCS)  57 

4.2.3  Dataset  3:  Depth  Acuity  Threshold  Measured  by  MoCS  for  Monocular  Observers. . . .  61 

4.3  Effects  of  Rod  Lighting  Model  Used  to  Render  Rod  Images  for  View  on  Devices. ...  66 

4.3.1  Unexpectedly  High  Stereoacuity  Thresholds  Observed  for  S3D  (“Laptop”)  Device...  66 

4.3.2  Follow-Up  Data  Collection  with  Lighting  Model  Changes .  68 

4.3.3  Lighting  Approaches  in  Seminal  Stereoacuity  Research  with  Physical  DPA  Devices  68 

4.4  Computer-Rendered  Images  Issues  for  the  Electronic  James  Depth  Acuity  Test .  69 

5.0  CONCLUSIONS  AND  RECOMMENDATIONS .  74 

5.1  Conclusions .  74 

5.2  Recommendations .  76 

6.0  REFERENCES . 77 

LIST  OF  SYMBOLS,  ABBREVIATIONS,  AND  ACRONYMS .  85 


ii 


LIST  OF  FIGURES 

Figure  Page 

1  eJames  Depth  Acuity  Experimental  Condition  at  6m .  2 

2  Close-up  of  Sample  Images  with  Both  10mm  Bars  at  Reference  Distance  (6m): 

(A)  Actuality  Integral  Slice  -  Perspecta  Spinning  Screen  Scanned  Volume  Display  (AIS); 

(B)  Stereo  3D  -  Dell  M6700  17.3-in.  Notebook  with  NVIDIA  Active -Eyewear  (S3D); 

(C)  Zebra  Integral  Ray  -  Hogel-Based  Zscape  Motion  Display  (ZIR); 

(D)  Depth  Perception  Apparatus  -  AFRL-automated  Howard  Six-Meter  Stereoscope  (DPA) ...  3 

3a  Quantitative  Metrics  for  Binocular  Single  Vision  (BSV)  Depth  Perception .  7 

3b  The  James  (1908  Depth  Perception  Apparatus  (DPA) .  8 

4  Howard  (1919a)  Version  of  the  James  (1908)  Depth  Perception  Apparatus .  9 

5  Howard  DPA:  Compact  Version  of  James  DPA  Designed  by  Howard  (1919b) 

for  Depth  Acuity  Testing  via  Two  Methods  (Forced  Choice,  Subject  Adjustment) ....  11 

6a  Photo  and  Drawing  of  the  Howard  (1919b)  Depth  Perception  Apparatus  Modified  as 

Described  in  Deyo  (1922);  “Howard-Dolman”  Should  Read  “Howard-Deyo” .  13 

6b  Original  James  (1908)  Depth  Test  Condition  as  Implemented  by  Deyo  (1922) .  14 

7a  Inside  View  of  AFRL-automated  USAFSAM  Howard  DPA .  19 

7b  Subject  view  of  Rods  in  AFRL-automated  Howard  DPA .  19 

8  Photo  of  Superimposed  Stereo  Pairs  Used  to  Assess  Pixel  Disparity .  22 

9  Measured  Pixel  Shifts  for  Modeled  Rod  Distances .  23 

10  Virtual  Model  Rod  Distance  Plotted  Against  Stereo  Displayed  Distance .  23 

11  Aircraft  Model  Displayed  in  2D  on  Flat  Panel  (left)  vs.  3D  on  AIS  Perspecta  (right)..  26 

12  Remote  Measurement  of  Rod  Images  on  AIS  Perspecta  using  Orthogonal  Laser 

Translation .  27 

13  Schematic  of  a  Hogel  within  a  Spatially  Integrated  Ray  (SIR)  Display .  29 

14  Photographs  of  Lightfields  Produced  from  a  3D  Model  Written  into  a 

Hologram  (left)  and  ZIR  9-Tile  ZMD  (right) .  31 

15  OpenCV  and  AutoHotKey  Inputs  Move  Cursor  to  Maximum  Intensity  Location .  34 

16  Hogel  Calibration  Pattern:  Initial  (Left),  Top  Two-Thirds  Realigned  (Right) .  35 

17  Calibration  Pattern  Appearance  after  Alignment .  36 

18  Optical  Set-Up  for  Estimating  Displayed  Image  Location  from  Secondary  Image .  38 

19  Scatter  Plot  of  Perceived  Depth  from  Commanded  Depth .  39 

20  Close-up  of  Four  3D  Devices  Under  Test  as  Arraigned  for  Subject  Evaluation .  40 

21  Experimental  Condition  for  Human  Subject  Evaluation  of  eJ  Methodology .  41 

22  Rendered  Bars  Designed  to  Have  10  mm  Width  at  Reference  Depth  on  S3D .  44 

23  Ill-defined  Edges  of  Bars  on  S3D  Laptop  Display  Due  to  Pixellation  Effects .  45 

24  Photograph  Array  of  the  Physical  Depth  Perception  Apparatus  (Howard  DPA)  Taken 
with  Camera  Located  6  m  from  Reference  Position  of  the  Fixed  Left  Rod  at  600  cm 

for  Right  Rod  Distances  (Top  to  Bottom)  of  606,  603,  600,  597,  and  594  cm .  46 

25  Photograph  Array  of  the  S3D  Display  (Dell  M6700  Laptop)  Bar  Pairs  Taken 
with  Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm)  for 

All  Combinations  of  0,  +3,  ±6  cm  Changes  in  the  Position  of  Either  Bar .  47 

26  Photograph  Array  of  the  Volumetric  (AIS  Perspecta)  Bar  Pairs  Taken  with 
Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm)  for 

All  Combinations  of  0,  ±3,  ±6  cm  Changes  in  the  Position  of  Either  Bar .  48 

iii 


27  Photograph  Array  of  the  Integral  Ray  (ZIR  1-Tile  ZMD)  Display  Bar  Pairs  Taken 
with  Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm)  for 

All  Combinations  of  0,  ±3,  ±6  cm  Changes  in  the  Position  of  Either  Bar .  49 

28  Method  of  Adjustment  (MoA)  Frequency  Histograms  of  Signed  Error .  50 

29  Method  of  Adjustment  (MoA)  Frequency  Histograms  of  Absolute  Error .  52 

30  Placement  Performance  vs.  Reference  Distance  with  Error  Bar  (±  1  SEM); 

Straight  Red  Line  Indicates  Optimal  Performance .  53 

31  Per  Trial  Placement  Performance  vs.  Reference  Distance;  Solid  Red  Lines  Show 

Optimal  Performance;  Dashed  Blue  Lines  Indicate  Near/Far  Misperceptions .  54 

32  Variability  of  Placement  Performance  vs.  Reference  Distance .  55 

33  Performance,  Averaged  Across  Viewers,  for  the  S3D  (“Laptop”),  AIS  (“Perspecta”), 
DPA  (“RealHD”),  and  ZIR  (“Zebra”)  Devices  Under  Test;  Error  Bars  Represent 

±1  SEM .  58 

34  Performance,  Averaged  Across  Viewers,  vs.  Depth  Stimuli  Separations  for  the 

S3D  (“Laptop”),  AIS  (“Perspecta”),  DPA  (“RealHD”)  and  ZIR  (“Zebra”)  Devices 
Under  Test;  Error  Bars  Represent  ±1  SEM .  59 

35  Indirect  Estimate  of  Threshold  Magnitudes  from  Data  Plotted  in  Figure  34: 

Intersections  of  80%  Correct  Performance  (Horizontal  Solid  Red  Line)  with  Device 
MoCS  Dataset  2  Performance  (Vertical  Dashed  Blue  Lines);  Error  Bars  ±1  SEM. . .  60 

36  Performance  vs.  Reference  Pin  Location;  Error  Bars  Represent  ±1  SEM .  63 

37  Performance,  Averaged  Across  Viewers,  vs.  Comparison  Bar  Location  for 

Reference  Bar  Fixed  at  600  cm;  Error  Bars  Represent  ±1  SEM .  65 

38  Close-up  Photographic  of  Front-Lit  Rod  Stimulus  Displayed  on  S3D  (“Laptop”). . .  67 

39  Close-up  of  Sample  Images  (Stimuli  Viewed  by  Subjects  at  6m)  on 

(A)  Physical  Depth  Perception  Apparatus  (DPA),  (B)  Active-Eyewear  Stereo  3D  - 
Samsung  22-in.  Notebook  Display  with  NVIDIA  Active-Eyewear  (aeS3D), 

(C)  Actuality  Rotating-Screen  Integral-Slice  Actuality  Integral  Slice  (AIS)  Perspecta 
Volumetric  Display  (ARS),  and  (D)  Zebra  Integral-Ray  Hogel-Based  Zscape  Motion 
Display  (ZIR) .  71 


IV 


Table 


LIST  OF  TABLES 


Page 


1  Means  and  Standard  Deviations  of  Absolute  Error  Performance,  across  Participants, 

for  Four  3D  Devices  Using  MoA  Dataset  1 .  55 

2  MoA  Stereoacuity  Threshold  Estimates .  56 

3  MoCS  Performance  Averaged  Across  Subjects  for  Four  Test  Devices .  57 

4  Estimated  Stereoacuity  Thresholds  Supported  by  the  Four  Devices  Under  Test 

Computed  Assuming  an  80.0%  Correct  Magnitude  as  Indicated  in  Figure  35 .  60 

5  Performance  across  Five  Subjects  when  Devices  Were  Viewed  Monocularly .  61 

6  Performance  across  Subjects:  Binocular  vs.  Monocular .  62 

7  Average  Binocular  vs.  Monocular  Performance .  64 


ACKNOWLEDGEMENTS 

The  authors  thank  Drs.  Lawrence  K.  Harrington  and  Paul  R.  Havig  II  for  arranging  for  the  loan 
of  the  USAFSAM  version  of  the  Howard  Depth  Acuity  Apparatus  (DPA)  used  in  this  work,  and 
for  providing  insightful  comments  during  the  early  phases  of  this  research. 


v 


1.0  SUMMARY 


Metrics  are  needed  for  the  Field-of-Light  Display  (FoLD)  class,  comprising  3D  visualization 
systems  that  provide  full-parallax  imagery  viewable  by  multiple  persons  without  special  eyewear 
(Hopper  et  al.,  2016).  The  present  work  explores  a  quantitative  metric  for  the  perception  of 
depth  on  electronic  3D  displays  at  6m  based  on  a  test  and  physical  apparatus  invented  by  James 
(1907,  1908),  an  ophthalmologist  working  in  the  UK,  which  were  subsequently  used  and 
advanced  by  other  research  ophthalmologists  in  The  Netherlands  (Schoiite  1910)  and  U.S. 
(Howard  1919a,  1919b)  (Deyol922).  The  6m  viewing  distance  is  chosen  so  as  to  enable 
correlation,  for  each  subject,  of  depth  acuity  to  lateral  acuity  measurements  per  Snellen  (1862). 

The  James  test  is  administered  on  a  physical  Depth  Perception  Apparatus  (DPA)  comprising  two 
parallel  vertical  white  rods  at  e.g.  6m  ±  z  viewed  through  a  black-fronted  window  (that  hides  the 
rod  ends)  against  a  black  background;  electric  lighting  is  used  to  ensure  uniform  illumination; 
head  position  is  controlled  to  eliminate  motion  parallax  (James,  1908).  Schoute  (1910)  reported 
results  for  560  persons  and  Howard  (1919a),  for  106  using  the  James  DPA.  Multiple  variants  of 
the  James  test  and  DPA  are  now  in  wide  use  for  disciplines  ranging  from  to  aviation  to  sports. 

The  electronic  version  of  the  James  test  (eJ)  developed  in  the  present  work  comprises  computer 
graphics  stimuli  designed  to  visually  emulate — within  limitations  of  the  electronic  displays — a 
variant  of  the  James  apparatus  designed  by  Howard  (1919b).  The  Howard  DPA  (a)  makes  the 
right  rod  hand-crank  adjustable  by  the  subject  (rather  than  the  administrator)  to  match  distance 
via  a  mechanical  string-pulley  system  and  (b)  enables  threshold  depth  acuity  measurement  by  the 
method  of  adjustment  as  well  as  the  method  of  forced  choice  (James  1908;  Howard  1919a). 

For  eJ,  a  wireless  hand-controller  replaces  the  hand-crank  to  automate  data  collection  and  subject 
trial  initiation.  Four  personal  computer  software  interfaces  control  the  stimuli  (bar/stripe) 
positions  presented  on  the  electronic  displays  and  on  an  AFRL- automated  version  of  the  Howard 
DPA  (Howard  1919b;  Meyer  et  al.,  2016).  The  systems  were  set  up  side-by-side  for  a  shake- 
down  evaluation  by  six  subjects  tested  via  two  methods:  (a)  forced-choice  of  nearest  bar;  (b) 
adjustment  of  one  bar  (randomly  positioned  in  depth  by  the  software)  to  match  the  other.  The 
design  and  programming  of  sample  images  for,  and  an  initial  human  subject  evaluation  on,  three 
electronic  3D  displays  are  reported  here.  These  three  comprise  one  Stereo  3D  (S3D)  class 
special  glasses  type  and  two  different  types  within  the  FoLD  class:  the  Actuality  Integral  Slice 
(AIS)  “Perspecta”  circular  rotating  screen  scanned  volume  display  (SVD);  and  the  Zebra  Integral 
Ray  (ZIR)  hogel-based  “Zscape  Motion  Display  (ZMD).”  The  AIS  Perspecta  and  ZIR  ZMD  (9- 
Tile  table,  1-Tile  mobile)  are  prototype  FoLD  systems  acquired  by  AFRL  in  2005  and  2011. 

Figure  1  shows  the  experimental  condition  of  subjects  viewing  the  displays  at  a  distance  of  6m 
from  the  reference  position  of  the  two  thin  circular  vertical  bars.  Visual  artifacts  (unique  to  each 
electronic  display  type)  impeded  creation  of  a  single  3D  graphical  representation  of  the  two-bar 
stimuli  and  complicated  cross-type  comparisons  (AIS  vs.  S3D  vs.  ZIR  vs.  DPA).  The  within- 
type  (i.e.  across  subjects)  and  limited  cross-type  analyses  (i.e.  across  displays)  reported  herein 
do,  however,  suggest  that  eJ  is  a  viable  depth  acuity  metric  to  guide  development  of  the  FoLD 
class  of  visualization  systems.  Figure  2  shows  a  close-up  of  the  stimuli  on  the  four  test  devices. 


1 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited.  88ABW  Cleared  09/08/2016;  88ABW-20 16-4404. 


Figure  1.  ejames  Depth  Acuity  Experimental  Condition  at  6m 
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The  stripes  representing  the  bars  on  the  electronic  displays  (AIS  in  Figure  2A,  S3D  in  Figure  2B, 
and  ZIR  in  Figure  2C)  are  written  as  white  against  a  black  background  per  James  (1907,  1908); 
the  eJ  test  images  are  electronic,  so  this  choice  can  be  varied  in  future  work.  The  DPA  (Figure 
2D)  has  black  bars  against  a  white  background  per  Howard  (1919a,  1919b).  The  eJ  bars  are  also 
scaled  (in  width  and  height)  as  a  function  of  their  displacement,  z,  from  the  reference  distance  of 
6m  so  that  they  subtend  constant  angles  at  the  pupil  positions.  The  physical  DPA  is  designed  to 
prevent  viewing  of  the  bar  ends.  Research  by  James  (1908)  and  Howard  (1919b)  shows  the  bar 
width  adjustment  has  an  insignificant  effect  on  the  depth  acuity  measured  for  observers.  Schoute 
(1910)  replaced  the  10mm  wooden  rods  with  “medium  thick  white  strings”  for  similar  reasons. 

Monocular  viewing  of  the  eJ  stimuli  are  also  tested  to  ascertain  the  effects  of  artifacts.  The 
present  results  are  consistent  with  Deyo  (1922)  who  reported  monocular  depth  acuity  is  worse 
than  binocular  acuity  by  10X  (4X)  for  subjects  having  20/15  (20/20)  Snellen  lateral  acuity. 

Howard  1919a)  reported  instant  perception  of  the  nearer  bar  via  binocular  single  vision  (BSV) 
using  a  shutter  to  rapidly  reveal  the  view  of  real  objects  (DPA).  Present  FoLD  MoCS  results 
(AIS  and  ZIR)  are  similarly  fast.  However,  stereoscopic  vision  (virtual  images)  and  BSV  (real 
objects)  are  not  the  same  (Blaauw,  1917,  p.  30).  The  inferiority  of  the  present  S3D  speeds  of 
depth  acuity  selection  (relative  to  the  AIS,  ZIR  &  DPA  results)  demonstrates  this  key  difference. 

Depth  acuity  metrology  will  have  to  be  expanded  in  future  work  to  address  multiple  viewing 
situations.  James  (1908),  Howard  (1919c,  1920b  on  Stereomicrometer);  and  Howard  (1919d, 
1920c  on  semaphores  at  100m)  examined  several  distances.  Group  viewing  may  involve  eye- 
display  distances  of  meters,  compared  to  arms-length  distances  for  workstations  and  handhelds. 


Figure  2.  Close-up  of  Sample  Images  with  Both  10mm  Bars  at  Reference  Distance  (6m): 

(A)  Actuality  Integral  Slice  -  Perspecta  Spinning  Screen  Scanned  Volume  Display  (AIS); 

(B)  Stereo  3D  -  Dell  M6700  17.3-in.  Notebook  with  NVIDIA  Active-Eyewear  (S3D); 

(C)  Zebra  Integral  Ray  -  Hogel-Based  Zscape  Motion  Display  (ZIR); 

(D)  Depth  Perception  Apparatus  -  AFRL-automated  Howard  Six-Meter  Stereoscope  (DPA) 
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2.0  INTRODUCTION 


2.1  Metrology  Gaps  for  Emerging  Field-of-Light  Display  (FoLD)  Systems 

The  AFRL  Battlespace  Visualization  Branch  (USAF  AFMC  711  HPW/RHCV)  has  acquired 
several  examples  of  a  new  class  of  displays  capable  of  producing  full-parallax  (FP)  3D  imagery 
that  do  not  require  eyewear  and  designed  for  view  by  several  persons  simultaneously  as  if  they 
were  looking  as  a  scaled  representation  of  a  physical  3D  model  or  scene.  Members  of  this  new, 
Field-of-Light  Display  (FoLD),  class  have  common  attributes  that  cannot  be  characterized  with 
existing  2D  display  test  and  evaluation  (T&E)  methods  and  standards.  New  metrics  and  testing 
procedures  are  required  as  detailed  in  the  AFRL  Technical  Report  entitled  “AFRL  Display  Test 
and  Evaluation  Methodologies  (AFTEM)”  (Hopper  et  al.,  2016).  A  methodology  for  depth 
acuity  at  a  viewing  distance  of  6  m  is  reported  here  to  advance  performance-based  assessments 
of  evolving  FoLD  technologies.  The  6m  reference  viewing  distance  was  chosen  by  Snellen 
(1862)  for  lateral  visual  acuity  and  adopted  by  James  (1908)  for  depth  visual  acuity,  which  is 
defined  as  the  threshold  angle  of  stereoscopic  parallax  (aka  stereo  acuity).  Future  work  will 
address  the  separate  viewing  conditions  of  1  m  (Howard,  1920a)  and  100  m  (Howard,  1920b). 

AFRL  is  uniquely  situated  to  lead  metrology  research  in  3D  visualization.  In-house  research 
addresses  both  physical  measurements  on  prototype  systems  and  human  subject  experiments 
with  the  aim  to  create  new  objective  physical  metrics  based  on  psychophysical  studies  of  the 
human  vision  system  (HVS).  The  FoLD  systems  currently  in-house  and  extramural  programs 
are  described  in  subsections  2.2  and  2.3.  High  resolution  fixed  holograms  are  available  as  well. 

2.2  FoLD  Systems  at  AFRL 

AFRL  presently  has  three  FoLD  systems  as  follows: 

(a)  a  volumetric  display — Acuality  Integral  Slice  (AIS)  Perspecta  swept  volume  system, 
which  computes  2D  circular  slices  of  a  3D  model  for  rapid  projection  via  a  TI DLP  XGA  kit 
onto  a  spinning  screen;  developed  by  Actuality  Systems  in  Burlington  MA,  bought  for  $75K 
during  the  2004-2008  period  (Reference:  Donelan,  2010).  (ARS  is  no  longer  in  production). 

Video  demo;  http://www. youtube. com/watch?v=8KaQmn2VTzs  (2007,  accessed  24  Jul  2016)) 

(b)  a  volumetric  display — LightSpace  Integral  Slice  (LIS)  Depth  Cube  zl024  system,  which 
computes  2D  depth  slices  of  a  3D  model  for  high  speed  projection  via  a  TI  DLP  XGA  kit 
into  a  stack  20  time-sequenced  LC -based  diffuser  screens  at  30Hz  to  generate  a  90°  FoV ; 

originally  developed  by  LightSpace  Technologies  (1994-2004,  price  ~$100K);  (July  2016  price:  $64.5K) 
re-developed  by  EUROLCDS,  SIA  (201 1)  as  new  x-series  LightSpace  xl403a,  replaced  by  model  xl405B. 
xl405B  available  from  LightSpace  Technology,  Inc.,  Twinsburg  OH,  www.lightspace3D.com  (2016). 

Video  demo:  http://www. youtube. com/watch?v=RAasdH10Irg  (5  Oct  2010).  (Reference:  Osmanis,  2016). 

(c)  a  hogel-based  display — Zebra  Integral-Ray  (ZIR)  ZScape  Motion  Display  system, 
which  computes  hundreds  of  millions  of  light  rays  from  hogels  of  a  3D  model  and  projects 
them  via  an  array  of  refractive  and  light-modulating  elements  into  a  90°  conical  viewing  zone 
at  ~1  Hz;  a  hogel  is  a  model  point  projected  into  NxM  directions  by  NxM  adjacent  2D  pixels, 
developed  by  Zebra  Imaging,  Inc.  in  Austin  TX.  (Reference:  Klug  et  al,  2013). 

(not  in  production;  prototype  ZMD1  systems  can  be  purchased  for  ~$1M). 

Video  demo:  http://www.zebraimaging.com/products/motion-displays  (accessed  24  Jul  2016) 
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2.3  AFRL  Programs  to  Develop  FoLD  Systems  and  Metrology 

AFRL  is  developing  performance  based  metrics  (PBMs)  for  FoLD  systems  as  part  of  both  its 
intramural  (e.g.  this  report)  and  extramural  programs.  The  extramural  3D  research  comprises 
primarily  SBIRs  sponsored  by  the  AFLCMC/HBB  Command  and  Control  Operations  Centers 
division  at  Hanscom  AFB  MA  and  AFLCMC/WNU  Human  System  Vison  at  WPAFB  OH.  The 
AFRL  extramural  program  builds  on  prior  investments  by  DARPA,  IARPA,  and  AFOSR. 

One  extramural  program,  Holographic  Lightfield  3D  Metrology  (HL3DM),  began  Phase  I  in 
2014  with  two  performers  (Adi-Displays  Consulting  LLC  in  Portland  OR  and  Rattan  Software 
LLC  in  Austin  TX)  with  a  planned  downselect  to  a  single  2-yr  Phase  II  performer  (Rattan)  from 
2016-2018.  The  HL3DM  objective  is  test  and  evaluation  methodology  for  holographic  lightfield 
3D  displays  with  an  automated  measurement  system  (a)  to  support  comparisons  of  prototypes 
emerging  from  research,  (b)  to  enable  robust  calibration,  and  (c)  to  perform  product  acceptance 
testing.  A  “Handbook  FoLD  Test  &  Evaluation  Methodologies”  is  anticipated  by  2018. 
Transition  via  incorporation  into  an  international  display  measurement  standard  is  also  planned. 

Another  program,  Holographic  Video  Display  (HVD)  focuses  on  SIR  systems.  One  is  a  Phase  II 
Extension  effort  2014-2018  with  TIPD,  LLC  (teamed  with  MIT  Media  Lab  and  Brigham  Young 
U.)  to  develop  a  new  type  of  LoLD  system,  the  hogel-based  TIPD  Integral  Ray  (TIR)  guided- 
wave  scanning  type,  along  with  metrics  to  assess  performance.  A  second  HVD  Phase  II  award 
2016-2018  to  Zebra  Imaging  Inc.  researches  a  next-generation  Zebra  Integral  Ray  (ZIR). 

ALRL  has  a  three-year  2014-2017  Cooperative  Research  and  Development  Agreement 
(CRADA)  with  Zebra  Imaging,  Inc.  entitled  “ZMD  Improvements/Analyses  and  Supporting 
Fixed  Hologram  Concepts ”  addressing  metrics  and  other  topics.  The  CRADA  leverages  the 
2005-2010  DARPA  Urban  Photonic  Sandtable  Display  (UPSD)  and  2010-2013  ALRL  Full 
Multiplex  Holographic  Display  (FMHD)  programs.  ALRL  is  the  DARPA  transition  partner  for 
USPD  and  acquired  the  ZIR  systems  and  spare  parts  from  DARPA  via  the  LMHD  program. 

ALRL  was  the  technical  and  contacting  agent  for  the  2012-2013  IARPA  Synthetic  Holographic 
Observation  (SHO)  with  Ostendo  Technologies,  Inc.  (OTI).  SHO  leveraged  the  2008-2011 
DARPA  High  Resolution  Affordable  Emissive  Micro-Display  (EMD)  and  the  2010-2011  ALRL 
Ultrahigh  Definition  Microdisplay  (UDM)  programs.  Ostendo  developed  its  Quantum  Photonic 
Imager  (QPI)  micro-pixelated  stacked  light  emitting  diode  (LED)  display  under  the  EMD,  UDM 
and  SHO  programs.  Sechrist  (2014)  documents  the  first  public  demonstration  of  a  QPI  device  at 
the  SID  Display  Week  2014  I-Zone  and  reports  that  Ostendo  plans  a  commercial  display  by  2015 
(did  not  happen).  An  array  of  QPI  devices  might  be  the  basis  for  a  hogel-based  Ostendo  Integral 
Ray  (OIR)  Quantum  Photonic  Imager  type  of  LoLD  system  once  the  technology  matures. 

2.4  Depth  Acuity  Metric  for  the  Real  World  -  the  James  Test  and  Distance  Stereoscope 

Depth  acuity  is  herein  defined  as  the  Binocular  Single  Vision  (BSV)  Depth  Perception  threshold. 
Binocular  Single  Vision  is  the  perception  of  a  single  image  of  real  objects  seen  with  both  eyes. 
Quantitative  metrics  for  the  depth  perception  threshold  can  be  defined  for  BSV  as  illustrated  in 
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Figure  3a.  Two  objects  A  and  B  are  at  distances  LA  and  LB  from  the  left  (L)  and  right  (R)  eyes. 
No  depth  difference  exists  if  both  objects  are  at  points  A  and  T  in  a  plane  parallel  to  the  inter 
pupil  distance  (IPD)  axis,  LR.  However,  if  the  object  at  T  is  moved  to  position  B,  the  line  of 
sight  LT  shifts  to  LB,  which  crosses  the  reference  plane  at  point  P.  The  lateral  linear  distance  PT 
is  a  quantitative  measure  of  depth  perception  called  binocular  Stereoscopic  Parallax  (Helmholtz, 
1856).  Another  linear  metric  is  the  depth  difference,  TB  =  z.  A  third  metric  is  the  angle  B-L-T, 
which  is  known  as  the  Angle  of  Stereoscopic  Parallax  (ASP)  (James,  1908).  The  threshold 
values  of  either  x  or  z  in  mm,  or  of  ASP  in  arcseconds,  are  quantitative  measures  of  depth  acuity. 

James  (1907,  1908)  invented  a  test  and  physical  apparatus  (Figure  3b)  that  he  called  a  “distance 
stereoscope,”  to  measure  stereoscopic  depth  visual  acuity  with  the  same  accuracy  as  the  Snellen 
(1862)  metric  for  monocular  lateral  visual  acuity.  James  carefully  excluded  visual  factors  other 
than  BSV  and  designed  a  test  and  apparatus  that  have  stood  the  test  of  subsequent  research. 
Figure  3b  is  an  annotated  reproduction  of  Fig.  2  in  James  (1908).  Fig.  1  in  James  (1908)  and 
associated  text  describes  the  James  DPA  as  a  380Wx310Hx385L  mm  box  with  roof  and 
sidewalls  removed;  the  back  included  two  190x290  mm  wings  that,  when  attached,  elongated  the 
back  wall  to  760  mm  wide.  James  used  vertical  rods  at  points  A  and  B  having  diameter  d±8  mm 
separated  by  a  distance  AB,  where  d=10  and  AB  =  50mm;  8  was  adjusted  from  0  to  0.5  mm  so 
that  the  rod  diameter  subtended  a  fixed  angle  of  about  6  arcmin  at  each  eye  for  LA  =  6  m.  The 
rods  were  painted  a  pure  white,  illuminated  by  two  electric  lamps,  and  viewed  at  Reference 
Distances  ranging  from  1850  to  6000  mm  through  a  195x120  mm  window  (centered  in  the  front 
wall)  against  the  back  wall;  the  whole  of  the  anterior  and  posterior  walls  were  covered  in  black 
velvet.  The  depth  acuity  of  subjects  was  found  to  vary  inversely  with  their  IPD  (James 
considered  values  from  50  to  70  mm).  Influences  of  rod  diameter,  illumination,  linear 
perspective  (rod  ends  hidden),  parallax  (from  slight  head  movements  at  6m),  and  accommodation 
are  excluded  and  the  observer  “must  fonn  an  estimate  as  to  their  relative  positions  from  the 
binocular  parallax  alone — i.e.  from  a  comparison  between  the  angles  subtended  at  the  nodal 
points  of  the  right  and  left  eye  respectively”  (James,  1908,  p.  1764).  James  also  performed  an 
experiment  in  which  the  rods  were  place  horizontally,  one  1 80  mm  farther  than  the  other;  in  this 
case  none  of  his  observers  was  able  to  ascertain  which  rod  was  nearer  at  viewing  distance  LA  as 
small  as  2.5m;  there  was  no  binocular  parallax  in  this  horizontal  rod  experiment  and  observers 
had  to  rely  solely  on  their  convergence  innervation.  James  reported  a  rough  correlation  of 
monocular  lateral  visual  acuity  (Snellen  test)  with  his  depth  acuity  results.  Applications  cited  by 
James  include  examinations  of  school  children  and  candidates  for  the  navy,  where  the  visual 
standard  required  for  both  eyes  is  high.  James  describes  his  Depth  Perception  Apparatus  as 
inexpensive  and  portable.  James  states  he  also  designed  a  small  instrument  on  the  same  principle 
to  be  used  at  reading  distance;  Howard  (1920a)  provides  design  details  for  a  Stereomicrometer. 

Results  using  the  James  DPA  are  similar  for  many  choices  of  the  parameters  (d,  8,  AT,  LA): 
(10mm,  0-0. 5mm,  50mm,  2-6m)  by  James  (1908);  (1mm,  0,  50mm,  2-5m)  by  Schoute  (1910); 
(10mm,  0,  60mm,  6m)  by  Howard, 1919a);  and  (120mm,  0-33mm,  lm,  100m)  by  Howard 
(1920b).  Schoute  (1910)  reports  that  depth  acuity  (the  threshold  BSV)  is  on  average  three  times 
as  sharp  (about  20  arcsec)  as  expected  based  on  Snellen  20/20  lateral  acuity  (1  arcmin).  Howard 
(1919a)  reports  thresholds  as  low  as  3.37  arcsec  using  his  version  of  the  James  DPA  (Figure  4). 

The  James  test  and  DPA  are  still  the  state-of-the-art  for  depth  perception  acuity  measurement 
(Lafayette,  2014)  and  a  suitable  basis  for  developing  a  depth  metric  for  electronic  3D  displays. 
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Quantitative  Metrics  for  Binocular  Single  Vision  (BSV)  Depth  Perception: 

(a)  Lateral  Linear  Stereoscopic  Parallax  (LLSP)  =  Line  Segment  PT  =  x; 

(b)  Angle  of  Stereoscopic  Parallax  (ASP)  =  Angle  P-L-T  e  Angle  B-L-T; 

(c)  Depth  Acuity  Metric  =  ASP  (arcsec),or  x  (mm), or  z  (mm). 

Notel:  Schematic  drawing  is  top  view  (not  to  scale)  of  Depth  Perception  Apparatus  (DPA)  per  James  (1908). 
Note  2:  Line  Segment  PT  =  LLSP  is  also  known  simply  as  Stereoscopic  Parallax  (SP)  per  Helmholtz  (1856). 
Note3:  Angle  B-L-T  s  ASP  is  also  known  as  the  Binocular  Parallactic  Angle  (BPA)  per  Howard  (1919a). 


Figure  3a.  Quantitative  Metrics  for  Binocular  Single  Vision  (BSV)  Depth  Perception. 
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The  above  figure  and  caption  are  Fig.  1  from  James  (1908)  illustrating  his  “Distance  Stereoscope”. 

Pure  white  sticks  are  used  as  test  objects;  all  other  surfaces  viewable  by  the  subject  are  covered  in  black  velvet. 
James  adjusted  the  diameter  of  the  sticks  to  maintain  constant  size  as  it  was  moved  from  z  =  0. 

Schulte  (1910)  used  this  same  apparatus,  but  with  medium  thick  white  threads  in  lieu  of  the  variable  diameter  sticks. 

Figure  3b.  The  James  (1908)  Depth  Perception  Apparatus  (DPA) 


Some  useful  equations  based  on  the  James  (1908)  diagram  that  served  as  the  basis  for  Figure  3a 
are  as  follows: 

ASP  (z)  in  radians  =  arctan(LR/RT)  -  arctan(LR/RB)  =  (IPD/RT)  *  (z  /  (  RT+  z))  Equation  (1) 
=  LR/RT  -  LR/RB  =  (LR  *  (RB-RT))  /  (RT  *  RB)  =  (LR  *  TB)  /  (RT  *  (RT  +  TB)) 

=  (LR/RT)  *  (TB  /  (RT  +  TB)) 

=  (IPD/RT)  *  (z  /  (RT+  z)) 

=  0.0105  *  z  /  (6000+z)  in  radians  for  (IPD,  RT)  =  (63mm,  6000mm) 

Deyo  (1922)  reported  the  ASP(z)  values  as  follows: 

=  32.54  urad  (6.7  arcsec)  for  (IPD,  RT)  =  (60mm,  6000mm),  z  measured  as  18.65mm  (BSV  all) 
=  20.82  urad  (4.3  arcsec)  for  (IPD,  RT)  =  (60mm,  6000mm),  z  measured  as  11.93mm  (BSVpass) 
=  202.5  urad  (41.7  arcsec)  for  (IPD,  RT)  =  (60mm,  6000mm),  z  measured  as  118mm  (mono) 

Qualified  pilots  averaged  a  z  threshold  of  11.93  mm,  which  translates  to  a  BSV  of  4.3  arcsec. 
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Fig.  1.  A  subject  making  an  observation. 


The  above  figure  and  caption  are  Fig.  1  from  Howard  (1919a)  illustrating  his  modification 
of  the  James  (1908)  apparatus  and  test  condition.  Flat  black  wooden  rods  are  used  as  test 
objects;  all  surfaces  are  painted  a  dead  black;  the  back  side  is  covered  with  a  sheet  of  dead 
white  cardboard.  The  diameter  of  the  rods  is  not  varied. 

Figure  4.  The  Howard  (1919a)  Version  of  the  James  (1908)  Depth  Perception  Apparatus. 
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Prior  to  1910  (and,  unfortunately,  still  to  this  very  day  in  much  of  the  world  literature)  it  was 
commonly  believed  that  the  smallest  distance  x=  PT  (threshold  ASP)  shown  in  Figure  3a  that 
could  be  perceived  corresponded  to  a  projected  distance  on  the  retina,  xretina,  equal  to  the  average 
cone  separation  (Helmholtz,  1856,  1910).  This  incorrect  belief  was  established  by  one 
researcher  (Helmholtz)  who  determined  his  own  binocular  threshold  to  be  xretina  =  4.4  pm,  the 
size  he  assumed  for  the  cones;  this  4.4  pm  retinal  distance  for  a  pupil-retinal  separation  of  15 
mm  corresponds  to  and  ASP  of  1  arcmin,  which  is  the  same  as  the  Snellen  standard  for  20/20 
lateral  visual  acuity.  Other  researchers  overlooked  the  lack  of  scientific  rigor  employed  by  the 
well-respected  Helmholtz  (just  one  biased  and  apparently  poor-sighted  self-observer)  and  it 
became  “mathematical  theory”  that  cone  pitch  determined  depth  acuity  and  lateral  acuity.  Work 
published  from  1910  through  1923  firmly  established  that  the  depth  acuity  was  much  smaller 
than  the  cone  size  of  an  individual  (Schoute  1910;  Blaauw  1917;  Howard  1919a). 

Blaauw  (1917)  summarizes  work  to  that  time  and  concludes  that  (a)  a  sharp  distinction  should  be 
made  between  BSV  (which  provides  different  lines  of  sight  fused  in  the  brain  depending  on 
attention  focus)  and  binocular  vision  using  a  stereoscope  (which  provides  a  fixed  lines  of  sight 
fused  in  the  optics  and  unchanged  by  mental  refocus  amongst  elements  in  the  field  of  view);  (b) 
good  lateral  visual  acuity  does  not  mean  also  good  appreciation  of  depth;  (c)  every  thorough 
examination  should  include  BSV;  and  (d)  the  ophthalmology  section  of  the  American  Medical 
Association  (AMA)  should  come  to  an  understanding  of  the  apparatus  to  be  used.  Blaauw  and 
other  ophthalmologists  paid  special  attention  to  pilot  selection  in  early  1917:  the  US  had  entered 
WW  I  and  in  April  1917  and  learned  that  90%  of  British  loses  the  first  year  of  the  war  were  due 
to  physical  defects  of  pilots,  not  defective  planes  (8%)  or  Germans  (2%).  An  independent  UK 
air  medical  service,  focused  on  care  of  the  flyer,  reduced  pilot-caused  loses  to  20%  in  two  years. 

The  US  Army  Air  Service  established  a  Medical  Research  Laboratory  (MRL)  on  19  Jan  1918  at 
Mineola  Long  Island  NY  to  develop  tests  for  selection,  from  amongst  an  oversupply  of 
applicants,  those  having  physically  distinguishing  abilities  for  success  as  combat  pilots,  to  create 
the  new  position  of  flight  surgeon  and  teach  aviation- specific  medicine,  and  to  investigate  field 
medical  issues  related  to  aircrew.1  For  some  18  months  from  Jan  1918  through  Jun  1919  the 
brightest  experts  in  various  medical  disciplines  were  commissioned  and  supported  with  extensive 
staffs  for  aviation  medical  research.  One  such  person  was  Dr.  Harvey  J.  Howard,  an  established 
excellent  research  ophthalmologist  assigned  to  MRL  and  tasked  with  establishing  if,  or  if  not,  a 
test  for  depth  acuity  should  be  used  in  pilot  selection.  Howard  (1919a)  used  the  James  DPA  and 
performed  extensive  experiments  to  establish  depth  acuity  (BSV  thresholds  ascertained  via  the 
psychophysical  method  of  forced  choice)  as  an  important  objective  test  for  pilot  selection. 
Howard  refers  to  the  ASP  in  Figure  3a  as  the  “Binocular  Parallactic  Angle  (BPA),”  and  reports 
thresholds  as  low  as  3.37  arcsec.  After  the  WWI  emergency  ended  on  11  Nov  1918,  Howard  and 
others  at  MRL  published  their  results  and  returned  to  private  practice  by  late  1919.  A  fewer 
number  of  civilian  researchers  were  brought  in,  but  teaching  aviation  medicine  became  the  focus. 


1  The  MRL  has  existed  continuously  since  its  establishment  in  1918  and  is  now  the  USAF  Air  Force  Material 
Command  711  Human  Performance  Wing  (711  HPW)  at  WPAFB  OH  comprising  the  same  research,  teaching  and 
extension  mission  in  the  form  of  its  Airmen  Systems  Directorate  (RH),  School  of  Aerospace  Medicine 
(USAFSAM),  and  Human  Performance  Integration  Directorate  (HP).  The  MRL  library  is  part  of  USAFSAM  library. 

10 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-2016-4404. 


2.5  Howard  Variation  of  the  James  Depth  Perception  Apparatus  (DPA):  The  Howard  DPA 

The  Howard  DPA  (see  Figure  5)  is  variation  of  the  James  DPA  that  simplifies  test  administration 
and  enables  depth  acuity  determination  via  the  psychophysical  method  of  adjustment  (by  the 
subject)  as  well  as  forced  choice.  Howard  (1919b)  exhibited  his  “Six-Meter  Stereoscope”  on  17 
Jun  1919  at  the  American  Ophthalmologic al  Society  (AOS);  a  conference  paper  is  published  in 
the  Nov  1919  Transactions  of  the  AOS  (TAOS)  and  a  more  detailed  paper,  in  the  Dec  1919  issue 
of  the  American  Journal  of  Ophthalmology  (AJO).  Fig.  1  in  Howard  (1919b)  and  associated  text 
describe  the  Howard  DPA  as  a  240Wx200Hx600L  mm  six-sided  box  with  two  compartments. 
The  back  200mm  compartment  contains  electric  lamps  behind  a  milk  glass  partition.  The  front, 
240Wx200Hx400L  mm  front  compartment  contained  two  180mm  long,  10mm  diameter  vertical 
rods  separated  by  60mm.  The  rods  were  painted  a  dead  black  and  viewed  at  Reference  Distance 
of  6000  mm  through  a  120Wxl00H  mm  window  (centered  in  the  front  wall)  against  the  back 
wall;  all  interior  walls  and  the  front  are  painted  a  dead  white.  Covering  the  window  from  behind 
is  a  Graflex  shutter  for  time  exposures  ranging  from  1  to  1000  ms.  For  the  method  of  adjustment 
a  cord/pulley  system  on  the  top  side  enabled  the  subject  to  adjust  the  rods  by  a  turning  knob  on 
the  pulley  wheel.  One  side  was  removable  for  adjustments  by  the  test  administrator. 

-Six  Mete^  Stereoscope 


Figure  5.  Howard  DPA:  Compact  Version  of  James  DPA  Designed  by  Howard  (1919b) 
for  Depth  Acuity  Testing  via  Two  Methods  (Forced  Choice,  Subject  Adjustment) 
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Modern  versions  of  the  Howard  DPA  shown  in  Figure  5  are  still  considered  state-of-the-art  for 
depth  acuity  measurement  and  are  widely  used.  For  example,  a  commercial  automated  depth 
perception  apparatus  is  available  $3,427.00  from  the  Lafayette  Instrument  Company  in  Lafayette 
IN.  The  Lafayette  Depth  Perception  Apparatus,  Model  14012A  is  a  closed  internally  LED- 
illuminated  box  with  dimensions  254Wx279Hx686L  mm,  rod  motion  accuracy  1mm,  power  10A 
for  1 10/220  60Hz  and  an  LCD  and  keypad  control  plus  digital  joystick  for  controlling 
displacement  (www.lafavetteevaluation.com/printview.asp?id=2141,  accessed  24  Jul  2016). 
Lafayette  states  that  “The  Depth  Perception  Apparatus  represents  the  state-of-the-art  in  sensation 
and  perception  measurement  technology”  and  is  used  to  test  depth  perception  acuity  for  pilots, 
crane  operators,  bus  drivers,  and  athletes  for  use  in  the  employment  selection  process.  The  DPA 
is  also  used  in  laboratory  research  and  teaching  demonstrations  (Lafayette  Instrument  Co.,  2016) 

2.6  Deyo  Modification  of  the  Howard  Six  Meter  Stereoscope:  The  Howard-Devo  DPA 

Deyo  (1922)  reports  measurements  she  attributes  to  a  Capt  Bascom  H.  Palmer  for  100  subjects 
using  a  DPA  she  describes  as  a  337Wx292Hx552L  mm  box  with  top  and  sides  removed  per 
James  (1908)  and  a  pulley  system  installed  per  Howard  (1919b),  but  on  the  bottom  side  rather 
than  the  top  side.  Deyo  states  that  the  entire  apparatus  is  painted  a  dull  black  and  the  anterior 
surface  of  the  back  covered  with  a  square  of  dead  white  cardboard.  Two  152mm  long,  9.5mm 
diameter  rods  are  viewed  through  a  197Wx89H  mm  window  (centered  in  the  front  wall). 
Following  James  (1908),  the  Deyo  DPA  is  placed  directly  before  and  beneath  the  Snellen  charts 
with  illumination  provided  for  both  by  one  daylight  lamp.  The  Howard-Deyo  DPA  and 
experimental  condition  are  illustrated  in  Figures  6a  and  6b.  An  average  depth  acuity  of  more 
than  z  =  30mm  (see  Figure  3a)  disqualifies  the  subject.  Deyo  (1922)  reports  threshold  depth 
differences  (z  in  Figure  3a)  for  100  cases  averaged  Zthreshoid  =  18.65  mm  (ASP  =  6.7  arcsec)  with 
both  eyes  vs.  Zthreshoid  =118  mm  (ASP  =  42  arcsec)  for  one  eye.2  The  sighting  eye  is  shown  to  be 
better  than  the  other,  and  subjects  with  20/15  to  have  a  significantly  smaller  z  (ASP  =  4.8  arcsec) 
than  subjects  with  20/20.  This  Howard-Deyo  DPA  is  still  widely  used. 

Some  confusion  exists  in  the  literature  regarding  the  Howard-Deyo  DPA  due  to  a  paper  by  Tefft 
and  Stark  (1922)  and  a  textbook  by  Bauer  (1926),  who  both  inserted  “Howard-Dolman”  as  an 
adjective  for  the  1922  Deyo-modification  of  the  1919  Howard  DPA  (Howard,  1919b).  There  is 
no  evidence  in  the  literature,  none  whatsoever,  of  any  publication  reporting  work  on  any  DPA 
apparatus  or  depth  perception  testing  by  anyone  named  Dolman.  Details  are  provided  below  in 
the  references  for  (Tefft  and  Stark,  1922)  and  (Bauer,  1926).  All  literature  references  to  a  so- 
called  Howard-Dolman  DPA  should  be  understood  to  refer  to  the  Howard  Six  Meter  Stereoscope 
DPA  (see  Figure  5)  modified  as  described  in  (Deyo,  1922).  Furthermore,  the  photographs 
published  as  Figures  1  and  2  in  Bauer  (1926)  pertain  to  the  descriptions  of  the  Howard-Deyo 
DPA  and  experimental  test  condition  described  textually  in  (Deyo,  1922).  Deyo  added  no 
adjectives  to  the  DPA  that  she  described  and  references  Howard  (1919a)  only.  The  names 
“Howard  DPA”  and/or  “Howard-Deyo  DPA”  should  replace  the  term  “Howard-Dolman  DPA” 
in  all  future  literature.  Furthermore,  the  term  Howard-Dolman  DPA  has  become,  erroneously, 
synonymous  with  any  of  dozens  of  versions  of  the  two  bar  test  invented  by  James  (1907,  1908); 
all  literature  references  to  this  two  bar  test  should  be  understood  to  refer  to  James  (1907,  1908). 


2  ASP  (z)  in  radians  =  arctan(LR/RT)  -  arctan(LR/RB)  =  (IPD/RT)  *  (z  /  (  RT+  z))  for  large  RT. 
ASP  (z)  in  arcsec  =  202265  *  (IPD/RT)  *  (z  /  (  RT+  z))  for  large  RT.  See  Figure  3  for  terms. 
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Fig.  1.  The  Howard-Dolma*  Depth  Perception-  Apparatos 


The  above  figure  is  a  photographic  plateinserted  in  Bauer(1926)  of  the  Depth  Perception  Apparatus 
described  by  Deyo  (1922).  The  List  of  Illustrations  in  thefrontmatterof  Bauer  (1926)  reads  "Fig.  I.The 
Depth  Perception  Apparatus"  and  Bauer  provides  no  reference  to  any  paper  by  any  person  named 
Dolman.  Bauer  does  reference  Howard  (1919a.  1919c.  1919d)  and  Deyo  (1922). 


FTii.  6. 

Howard- Dolman  Depth  Perception  Apparatus.  a,  slide  carrying  adjustable 
rod:  b.  millimeter  scale;  r.  stationary  rod;  </,  adjustable  rod;  e,  aperture  through 
which  rods  are  viewed;  /,  cords  for  adjusting  rod;  <?f  modified  method  of  illum¬ 
ination  consisting  of  cabinet  faced  with  ground  glass  behind  which  is  50  watt 
bulb. 

The  above  figure  is  a  photographic  schematic  drawing  inserted  in  Thome(1928)  of  the  Depth  Perception 
Apparatus  described  by  Howard  (1919b)  with  thetop  and  sides  removed.  Thome  references  Howard 
11919a)  but  provides  no  referenceto  any  paperbv  anv  person  named  Dolman. 

Figures  6a.  Photo  and  Drawing  of  the  Howard  (1919b)  Depth  Perception  Apparatus 
Modified  as  Described  in  Deyo  (1922);  “Howard-Dolman”  Should  Read  “Howard-Deyo”. 
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Fio.  2.  Testing  Depth  Perception 

The  subject  is  seated  20  feet  distant  from  the  rods 

NOTE:  The  above  figure  is  a  photographic  plate  inserted  in  Bauer(1926|  of  thetest 
condition  andthe  Depth  Perception  Apparatus  described  in  Deyo  (1922) 

Figure  6b.  James  (1908)  Depth  Test  Condition  as  Implemented  by  Deyo  (1922). 
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Thorne  (1928)  includes  a  schematic  diagram  (see  Figure  6a,  bottom)  of  the  DPA  described  in 
Deyo  (1922)  and  states  that  it  is  used  as  an  aviation  vision  test  by  accepting  applicants  whose 
depth  acuity  =  threshold  ASP  =  9. 1 19  arcsec  or  less,  which  for  an  average  American  IPD  =  64 
mm  requires  z  =  25  mm  or  less. 

2.7  Speed  and  Accuracy  of  the  James  Test 

The  James  apparatus  built  in  Mineola  by  Howard  is  described  by  visitors  as  having  extreme 
precision,  with  a  very  slight  variation  of  in  the  distance  of  between  objects  seen  at  a  distance  of 
6m  being  instantly  perceptible  to  the  observer  (Howard,  1919a,  discussion  following  14  Jun 
1919  presentation  by  Howard  at  55th  Meeting  of  the  American  Ophthalmologic al  Society;  see 
Transactions  of  the  American  Ophthalmological  Society  (TAOS)  17,  755-757,  Nov  1919). 

Andersen  and  Weymouth  (1923)  propose  a  theory  to  account  for  the  fineness  of  depth  acuity 
based  on  a  complex  percept  of  position  that  they  propose  to  call  retinal  mean  local  sign  (RMLS) 
and  state  that  factors  contributing  to  RMLS  include: 

“7,  the  mutual  effect  of  adjacent  retinal  elements  in  either  retina; 

2,  the  averaging  of  the  successive  stimulus  patterns  on  each  retinal  mosaic  caused  by  the 
constant  slight  eye-movements;  and 

3,  the  combining  of  the  two  simultaneous  stimulus  patterns  presented  to  the  two  eyes.” 

“As  a  result  of  these  three  factors  the  mean  local  sign  (in  the  present  case  of  the  images  of 
the  [two]  threads)  may  be  so  accurate  that  a  disparity  between  images  of  a  fraction  of  a 
cone-diameter  may  be  correctly  perceived.” 

Empirical  evidence  has  shown  that  “may  be  correctly  perceived”  should  read  “will  be  correctly 
perceived  by  a  normal  sighted  person”  (James  1908;  Schuler  1910;  Howard  1919a;  Devo  1922). 

Hirsch  and  Weymouth  (1948a,  1948b)  expanded  on  Andersen  and  Weymouth  sought  to  combine 
certain  aspects  of  physiology  with  the  methods  of  psychophyics. 

2.8  Limitations  of  the  James  Test  for  Understanding  Image  Perception  on  3D  Displays 

Zalevski  et  al.  (2007)  summarize  depth  acuity  (aka  stereoacuity)  research  from  1900  through 
2002  and  note  that  the  lowest,  2-6  arcsec,  depth  acuity  reported  for  a  stimulus  comprising  two 
parallel  vertical  lines  horizontally  displaced  (i.e.  the  James  test)  deteriorates  by  factor  of  seven  or 
more  for  a  stimulus  comprising  a  square  (same  two  vertical  lines,  but  with  two  horizontal  lines 
connecting  the  tops  and  bottoms).  Thus,  the  perceptual  value  of  3D  displays  is  also  a  function  of 
the  image  composition  in  practical  applications.  This  issue  should  be  examined  in  future  work. 

2.9  S3D  Systems  at  AFRL 

AFRL  has  multiple  variants  of  the  Stereo  3D  Display  (S3D)  class  of  visualization  system. 
However,  most  laboratory  and  operational  visualization  applications  opt  for  2.5D — 3D  models 
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interactively  rendered  on  ever-higher  resolution  2D  displays — due  to  the  multiple  hardware  and 
perceptual  limitations  of  all  S3D  systems.  Problems  limiting  wider  adoption  of  the  S3D  class 
include  the  (a)  objection  of  most  users  in  most  applications  to  the  special  eye-selection  wear,  (b) 
nausea  after  just  a  few  minutes,  and  (c)  failure  to  produce  full  parallax  views  with  correct  BSV 
for  each  viewer.  All  viewers  of  S3D  see  the  same  viewpoint  and  focus  distance  (for  all  pixels) 
regardless  of  their  position  or  head  orientation  in  the  viewing  zone.  Donning,  doffing,  loosing, 
finding,  and  tolerating  the  special  S3D  glasses  (e.g.  to  see  other  persons  or  2D  screens  at  full 
brightness  results)  has  driven  most  usage  to  episodic  scenarios  (e.g.  sessions  of  10  min  or  less). 

There  are  some  important  types  within  the  S3D  class  that  attempt  to  address  the  aforementioned 
objections:  head-tracked  (HTS3D),  active-retarder  head-tracked  (ARHTS3D),  stylus-controlled 
(SCS3D),  and  combined  stylus  controlled,  active-retarder,  head-tracked  (STARTS3D).  The 
HTS3D  type  adds  head-tracking  to  S3D  along  with  software  and  fast  graphical  processing  to 
render  and  update  the  left/right  views  to  those  appropriate  to  the  instantaneous  eye  positions.  The 
ARHTS3D  type  integrates  an  active  retarder  into  the  HTS3D  screen  housing  to  enable  the  use  of 
passive  polarization  glasses  to  retain  the  full  native  resolution  of  the  display  (active  glasses 
typically  reduce  horizontal  resolution  seen  by  half).  The  STARTS3D  type  adds  a  separately 
tracked  stylus  to  the  ARHTS3D  to  achieve  a  notable  trifecta:  dynamic  user-interaction  with  3D 
data  and  models  with  head-motion  parallax.  The  first  member  of  the  STARTS3D  class  is  the 
zSpace  Immersive  Reality  System  (www.zSpace.com).  AFRL  has  two  zSpace  displays. 

The  Auto-Stereoscopic  3D  Display  (AS3D)  class  of  visualization  system  is  not  used  in  the 
present  study  but  is  mentioned  for  completeness.  The  AS  3D  class  provides  multiple  horizontal 
viewing  zones  that  can  present  adjacent  perspectives  of  a  3D  scene  (data  set,  model);  up  to  200 
zones  have  been  demonstrated  and  up  to  16  are  offered  as  consumer  products.  The  AS3D  class 
do  NOT  require  the  special  eyewear,  but  provide  horizontal  parallax  only  and  have  objectionable 
discontinuities  between  the  viewing  zones.  A  head-tracked  type  (HTAS3D)  in  which  helmet 
position  is  tracked  and  the  left/right  views  dynamically  moved  to  keep  the  correct  view  on  the 
correct  eye  as  the  viewer  moves  is  available.  The  first  electronic  HTAS3D  is  the  one  developed 
by  Dimension  Technologies,  Inc.  (www.dti3D.com)  under  SBIR  funding  from  AFRL  in  the  early 
1990s  and  has  recently  been  awarded  a  Phase  II  contract  by  NASA  to  further  mature  their 
glasses-free  cockpit  3D  technology.  Several  modern  versions  of  this  25-year  old  HDAS3D  are 
available  (see  e.g.  Surman  et  al.  2006). 

2.10  Psychophysical  Research  on  3D  Displays 

Psychophysics  quantitatively  investigates  the  relationship  between  physical  stimuli  and  the 
sensations  and  perceptions  they  effect.  In  this  study  it  is  posited  that  the  defining  characteristic 
of  depth  perception  should  be  a  key  metric  for  the  characterization  and  comparison  of  various 
classes  and  types  of  3D  display.  The  specific  psychophysical  measurement  selected  here  is  the 
James  (1908)  two  vertical  bar  test.  Lloyd  (2012)  applies  this  test  to  S3D  display  with  graphically 
generated  bars  as  the  stimuli  and  reports  that  “disparity  thresholds  (5  to  10  arcsec)  are  obtainable 
on  electronic  [S3D]  displays  with  pixel  pitch  [at  the  eye  nodal  point]  as  coarse  as  2  arcmin,  but 
only  if  sufficient  antialiasing  is  applied.”  Thus,  significant  attention  is  required  in  the  design  and 
programming  of  the  electronic  versions  of  the  graphical  stimuli  for  the  electronic  James  (eJ)  test. 
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3.0  METHODS,  ASSUMPTIONS  AND  PROCEDURES 


3.1  Software  Suite  for  Graphical  Stimuli  Emulating  the  James  DPA  Sticks 

The  eJames  Software  Suite  (eJSS)  developed  for  the  present  effort  includes  all  the  code  required 
to  produce  a  virtual  model  of  a  typical  physical  two-vertical-stick  (two-bar)  James  test  and  the 
necessary  interfaces  to  set-up  and  run  automated  evaluations  as  illustrated  in  Figure  1.  In  the 
present  report  the  eJSS  is  applied  to  an  AFRL- automated  physical  Howard  DPA,  and  to  three 
types  of  electronic  3D  displays:  stereo,  volumetric,  and  integral-ray. 

The  software  suite  uses  OpenSceneGraph,  a  high  performance  application  programing  interface 
(API),  to  generate  3D  objects  in  OpenGL  (a  lower  level  API),  resulting  in  the  desired  virtual 
model.  The  eJSS  stimulus  generation  interface  allows  the  researcher  to  adapt  the  model  for  a 
specific  display  and  experimental  design.  A  wide  variety  of  design  variables  are  supported 
including  subject  viewing  distance,  subject  inter-pupillary  distance,  display  type  (stereoscopic  or 
volumetric),  physical  display  size/volume,  stimulus  lighting  model,  stimulus  duration,  number 
of  trials,  trial  initiation  cue,  inter-trial  rest  period.  Psychophysical  methods  supported  include 
methods-of-adjustment  (MoA),  staircase,  and  constant-stimuli  (MoCS),  aka  forced-choice. 

The  eJSS  interface  to  the  AFRL-automated  physical  Howard  DPA  uses  the  virtual  model  to 
generate  a  set  of  software  commands  that  an  Arduino  microcontroller  (www.arduino.ee, 
http://en.wikipedia.org/wiki/Arduino)  translates  into  low  level  electric  motor  commands.  These 
microcontroller  commands  configure  the  physical  Howard  DPA  to  match  the  virtual  model. 

The  eJSS  interface  to  the  stereo  display  uses  two  virtual  cameras  to  render  the  model  into  stereo 
bit  maps  appropriate  for  the  targeted  display  and  subject  viewing  geometries.  Full  screen 
antialiasing  is  supported  by  the  use  of  NVidia  Quadra  video  cards. 

The  eJSS  also  allows  direct  access  to  the  model  by  display  systems  that  ingest  (handle)  OpenGL 
data.  Examples  in  the  present  work  are  the  Actuality  Integral  Slice  (AIS)  circular  rotating-screen 
Perspecta  volumetric  display  and  the  Zebra  Integral  Ray  (ZIR)  hogel-based  Zscape  Motion 
Display  (ZMD).  Subjects  perform  the  experimental  task  through  a  standard  video  game 
controller.  At  the  end  of  each  experimental  trial  the  subject  response  is  automatically  exported  to 
the  data  collection  spread  sheet. 

The  James  test  is  described  in  the  introduction  (see  Figure  3  and  associated  text).  The  Howard 
(1919b)  variant  of  the  original  James  Depth  Perception  Apparatus  (DPA)  enabled  the 
psychophysical  method  of  adjustment  (MoA)  in  addition  to  the  forced-choice  method  (MoCS). 
Both  methods  are  automated  by  the  eJSS.  In  addition,  the  software  enables  (a)  scaling  of  the 
stimuli,  (b)  arbitrary  variation  of  the  position  of  both  bars,  (c)  and  choice  of  white  bars  on  black 
background  per  James  (1908)  or  black  bars  on  white  background  per  Howard  (1919a,  b,  c,  d). 

All  data  collected  in  this  study  uses  the  James  choice  of  white  bars  against  a  black  background 
for  the  electronic  displays  (see  Figure  2A,  2B,  2C).  The  DPA  used  is  difficult  to  photograph  (see 
Figures  1,  2)  with  the  electronic  displays  due  to  the  all-white  interior  surfaces  and  length-of-box 
interior  lamps  used  by  USAFSAM  (see  Figure  7a)  for  its  in-house  variant  of  the  Howard  DPA. 
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3.2  Display  Class  &  Type:  Physical  Apparatus  &  Two  Vertical  Sticks 


The  James  (1907,  1908)  depth  test  is  maximally  dependent  on  instantaneous  binocular  parallax. 
Monocular  depth  cues  are  reduced  to  the  greatest  extent  possible.  Two  vertically  oriented  sticks 
(aka  strings,  rods,  dowels,  bars)  are  viewed  through  a  window  to  hide  their  ends  to  eliminate  the 
perspective)  cue.  The  sticks  are  presented  in  maximum  contrast  to  the  background  to  reduce 
lighting  variations.  A  6m  viewing  distance  of  narrow  10mm  (or  less)  diameter  sticks  is  common 
to  limit  accommodation  and  vergence  cues.  The  observer  is  advised  to  keep  their  head  steady 
(no  large  head  motions  during  testing)  to  eliminate  motion  parallax  as  a  significant  cue.  The 
observer  is  not  shown  the  sticks  during  the  time  they  are  being  moved  to  their  initial  position  for 
an  experimental  trial  to  enable  stationary  stimulus  presentation.  Observers  typically  perceive 
depth  differences  rapidly  before  accommodation  can  occur. 

3.2.1  Device  Under  Test:  AFRL  Computer- Automated  Howard  DPA 

A  DPA  based  on  the  design  published  by  Howard  (1919b)  was  built  by  personnel  at  the  USAF 
AFMC  711  HPW  United  States  School  of  Aerospace  Medicine  (USAFSAM).  The  USAFSAM 
physical  apparatus  is  a  six-sided  260x260x800mm  (inside)  wooden  box  painted  white  inside  and 
black  outside,  with  interior  illumination  provided  by  two  fluorescent  bulbs  that  run  along  the  top 
of  the  800mm  inside  walls  (see  Figure  7a).  Two  500mm  rails,  parallel  with  a  50mm  center-to- 
center  spacing,  straddle  the  centerline  of  the  floor.  On  top  of  each  rail  sits  a  10mm  diameter 
vertical  black  rod.  The  left  rod,  as  seen  from  the  viewing  window,  is  fixed  in  position  half-way 
down  its  rail;  the  apparatus  is  typically  placed  such  that  the  fixed  rod  is  6m  from  the  eye  nodal 
points  of  the  subject.  The  right,  travelling  rod  is  adjusted  by  a  loop  of  cable  running  from  the 
observer  through  holes  in  the  front  of  the  box,  through  the  base  of  the  right  rod,  and  around  a 
pulley  at  the  back  of  the  box.  A  bent  wire  attached  to  the  base  of  the  movable  rod  passes  through 
a  horizontal  slot  in  the  right  side  to  indicate  position  on  a  scale  mounted  on  the  outside.  A 
160x80  mm  viewing  window  centered  in  the  front  end  limits  the  view  of  the  observer  to  just  the 
middle  of  the  two  rods  (see  Figure  7b).  An  experiment  requires  the  test  administrator  to  cover 
the  window,  move  the  right  rod,  and  remove  the  cover;  the  observer  then  declares  which  is  closer 
(method  of  forced  choice  using  constant  stimuli)  or  uses  the  cable  to  adjust  the  right  rod  until  the 
depths  are  perceived  to  match  (method  of  adjustment). 

The  711  HPW/USAFSAM  Howard  DPA  was  manually  operated,  which  slowed  clinical  use  and 
severely  limited  the  potential  for  use  in  human  perception  research.  The  AFRL  Battlespace 
Visualization  Branch  (711HPW/RHCV)  agreed  to  automate  this  device  in  exchange  for  its  use  in 
electronic  3D  display  research. 

Key  automation  features  installed  by  AFRL  711  HPW/RHCV  in  the  physical  apparatus  include: 
(a)  rotary  solenoid  actuated  shutters  to  occlude  the  window  under  computer  control;  (b)  a 
photoelectric  switch  for  use  in  system  calibration;  and  (c)  an  Arduino  microcontroller  to  translate 
directions  from  the  eJ  software  into  executable  commands  for  the  electric  motors.  The 
microcontroller  also  enabled  automated  data  collection  by  reporting  the  position  of  the  movable 
rod  back  to  the  eJ  software.  The  addition  of  a  solenoid  and  microcontroller  necessitated 
supporting  infrastructure  improvements  including  shielding/grounding  of  power  circuits  and  the 
installation  of  a  cooling  fan.  A  report  on  the  AFRL-Howard  DPA  is  planned  (see  Meyer  et  al.). 
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Figure  7a.  Inside  View  of  AFRL-automated  USAFSAM  Howard  DPA 
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The  interior  view  in  Figure  7a  shows  the  belt  drive  and  stepper  motor  that  are  now  in  the  device 
(replacing  the  cable  and  pulley).  Electronically  controlled  doors  occlude  the  subject  view  while 
computer  random  number  generator  commands  and  moves  the  right  rod  to  its  next  position.  For 
testing  using  the  method  of  adjustment,  the  doors  fly  open  under  computer  control  and  the 
subject  adjusts  the  position  of  the  right  rod  using  a  computer  game  controller  whose  inputs  are 
processed  by  the  eJSS  and  sent  via  the  Arduino  microcontroller  to  the  stepper  motor.  For  testing 
using  the  method  of  constant  stimuli  (forced  choice),  the  subject  simply  presses  a  left  or  right 
button  on  the  hand-controller  to  indicate  which  rod  is  perceived  to  be  closer.  Both  methods  can 
be  used  to  determine  the  just  noticeable  difference  (JND)  depth,  or  threshold  depth  acuity,  of  the 
subject. 

3.2.2  Calibration  of  the  AFRL-  Automated  Howard  DPA 

The  motor  step  size  is  calculated  by  moving  the  rod  a  measured  distance  and  dividing  by  the 
number  of  steps  incurred.  An  infrared  FED  and  photo-electric  switch  is  installed  on  opposite 
sides  of  the  right  rail,  at  the  limit  of  the  forward  most  rod  position.  Each  time  the  system  goes 
through  the  power-up  sequence  the  stepper  motor  automatically  moves  the  rod  forward  until  it 
activates  the  photo-electric  infrared  FED,  thereby  defining  the  reference  position  (i.e.  the 
forward-most  position)  for  the  right  rod.  Once  is  reference  position  is  defined  the 
microcontroller  keeps  a  running  count  of  the  number  of  motor  steps  between  the  reference 
position  and  current  location.  The  current  rod  location  could  then  be  calculated  by  multiplying 
the  number  of  steps  by  the  step  size  in  mm.  Confirmation  of  system  calibration  is  accomplished 
by  using  the  stimulus  generation  interface  of  the  eJSS  to  generate  a  series  of  experimental  trials. 
When  the  trials  are  run  the  starting  location  of  the  virtual  model  is  compared  to  the  starting 
location  of  the  rod  in  the  physical  Howard  DPA.  Trials  terminate  with  the  rod  in  a  variety  of 
locations  distributed  throughout  the  length  of  the  right  rail.  Final  right  rod  locations  are 
compared  to  the  recorded  position  in  the  automated  data  collection  spread  sheet  and  a  lookup 
table  is  generated.  Positioning  accuracy  of  <lmm  is  confirmed  for  the  AFRF- Automated 
Howard  DPA. 
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3.3  Display  Class  &  Type:  S3D  &  Eye  Selection  Glasses  (ESG) 


Stereo  3D  displays  (S3D)  attempt  to  add  image  disparity  to  monocular  cues.  The  appeal  of 
stereo  is  often  described  as  “intuitive,”  but  it  has  long  been  known  that  S3D  is  NOT  a  sufficient 
approximation  of  real  world  3D  vision  (Blaauw,  1917).  The  subtle  differences  between  left/right 
images  captured  by  cameras  or  generated  by  computers  (full  images  at  a  fixed  focal  distance  and 
view-direction)  are  inadequate  to  fully  generate  the  perception  of  depth  (binocular  stereopsis).  In 
practice,  the  perception  created  by  S3D  is  dramatic  but  unpredictable  from  person  to  person  and 
from  scene  to  scene.  And  for  over  100  years  there  is  a  widely  reported  frequent  discomfort 
associated  with  stereo  viewing.  Stereo  cues  provide  neither  shape  constancy  nor  a  quantitative 
depth  perception  metric  (Westheimer,  2011;  Johnston,  1991;  Scarfe  and  Hibbard,  2008).  These 
failures  are  caused  the  inability  of  S3D  to  represent  a  consistent  and  complete  set  of  depth  cues. 
At  least  part  of  the  discomfort  reported  is  due  to  conflicting  sensory  cues  for  accommodation  and 
vergence  (Blaaw,  1917;  Hoffman  et  al.,  2008).  Other  sensory  conflicts  may  also  be  important 
including  binocular  parallax-motion,  blur-disparity,  binocular  optical  asymmetries,  or  higher- 
level  cognitive  conflicts  (Patterson,  2009). 

3.3.1  Rendering  for  Temporally  Multiplexed  Left/Right  Eye  S3D  Perspective  Views 

Stereo  rendering  is  based  on  a  pair  of  virtual  cameras.  The  virtual  camera  used  in  graphic 
rendering  is  analogous  to  a  photographic  camera  capturing  a  real  scene.  In  both  types  of 
cameras,  the  camera  position  defines  how  the  monocular  3D  perspective  cues  will  be  rendered. 
The  magnification  or  zoom  setting  determines  the  field  of  view  and  focus  distance.  The  spatial 
relationship  between  the  virtual  camera  and  the  lighting  model  creates  shadows  and  highlights. 
Since  a  virtual  camera  is  not  burdened  by  optical  limitations,  depth  of  field  is  not  a  basic  attribute 
of  virtual  cameras.  However,  a  depth  of  field  function  can  be  incorporated  into  the  rendering 
algorithm  if  desired.  When  directly  viewing  an  object,  motion  parallax  is  derived  from  the 
object  location  and  the  observer  location.  Moving  the  camera  captures  motion  parallax  cues, 
however,  in  graphics  the  monocular  depth  cues  are  all  determined  by  the  virtual  camera  settings 
and  its  position  in  the  model  space.  Furthermore,  the  camera-captured  cues  are  independent  of 
the  variable  positioning  of  the  observer  relative  to  the  display.  It  is  possible  to  align  the  virtual 
cameras  and  eye  positions  by  (a)  fixing  the  observer  or  (b)  monitoring  their  movement  and 
updating  the  rendering  viewpoint  accordingly.  Rendering  by  virtual  camera  produces  a  2D 
bitmap  comprising  a  set  of  digitally  encoded  intensity  values  across  an  array  of  locations.  There 
are  significant  analogies  between  stereo  rendering  and  stereo  photography:  the  boundaries  of 
good  stereo  practice  are  still  defined  by  the  limits  of  the  human  visual  system.  Stereo  rendering 
increases  the  graphics  processing  over  that  required  for  2D  to  a  relatively  small  extent.  Real 
time  stereo  rendering  at  high  definition  spatial,  temporal,  and  color  resolutions  is  not  a  challenge 
for  current  computer  technology. 

3.3.2  Display  Under  Test:  Dell  M6700  S3D  System  with  Liquid  Crystal  Glasses 

From  2012  to  2013  Dell  produced  a  professional  class  S3D  notebook  computer  system  based  on 
a  high  end  M6700  laptop  that  integrated  a  17. 3 -in.  120  Hz  UltraSharp™  active  matrix  liquid 
crystal  display  (AMLCD),  an  NVidia  3D  Vision  Pro  S3D  kit,  and  a  4  GB  NVidia  Quadro 
K4000M  discrete  graphics  card.  The  native  resolution  of  the  AMLCD  panel  was  1920 
(horizontal)  x  1080  (vertical)  with  a  screen  size  of  381  mm  (horizontal)  x  215  mm  (vertical).  The 
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system  requires  the  wearing  of  electro-optical  liquid-crystal  active  shutter  glasses  that  rapidly 
oscillated  between  transparency  and  opacity  in  synchrony  with  the  display  oscillation  between 
imagery  for  each  eye  (at  60  Hz  per  eye). 

3.3.3  Calibration  of  S3D  Display 

The  stimulus  generation  software  interface  was  used  to  create  a  model  of  the  physical  DPA  rods 
for  stereo  rendering.  A  viewing  geometry  for  the  stereo  virtual  cameras  was  defined  and 
expected  left/right  eye  image  seperations  were  calculated  for  a  series  of  rod  positions  using  well- 
established  retinal  disparity  equations.  A  general  discussion  of  these  equations  can  be  found  in 
publications  by  Cormack  and  Fox  (1985,  1986),  wherein  the  term  “half  image”  is  used  for  the 
left-  and  right-eye  images  which  were  both  were  printed  on  the  same  page  (in  different  colors) 
for  their  work.  The  half  image  separations  were  then  divided  by  pixel  pitch  to  calculate  the 
expected  pixel  shifts  for  the  stereo  images  pairs.  The  predicted  shifts  were  compared  to  measured 
shifts  from  the  displayed  stereo  image  pairs.  To  measure  the  shifts,  high  resolution  digital 
photographs  of  the  rod  images  superimposed  with  a  millimeter  rule  were  taken  and  visually 
analyzed  as  pictured  in  Figure  8.  In  the  measurement  trials,  the  left  and  right  rods  were  stepped 
through  the  series  of  depth  locations  in  unison  in  order  to  verify  uniformity  of  display  depth 
response  for  both  rods.  Anti-aliasisng  was  deactivated  during  these  measurements  to  improve  the 
readability  of  the  photographs,  as  a  result  display  response  in  pixel  shift  was  a  step  function. 
Assessment  of  pixel  shift  was  accomplished  by  counting  the  number  of  pixels  between  the  edge 
of  the  rod  image  for  the  stereo  pairs.  Since  the  step  from  one  pixel  shift  increment  to  the  next 
did  not  occur  simultanously  for  both  the  right  and  left  rod  edges,  both  sides  of  the  rod  had  to  be 
measured.  The  period  between  the  first  edge  shift  and  the  second  edge  shift  was  considered  a 
half  pixel  shift.  The  measured  pixel  shift  as  a  function  of  the  modeled  viewing  distance  is 
graphed  in  Figure  9.  These  measured  pixel  shifts  were  converted  to  depths  using  the  retinal 
disparity  equations.  The  modelled  rod  depth  is  plotted  against  the  displayed  rod  depth  in  Figure 
10;  a  slope  of  1.02  is  computed  by  regression  analysis  with  a  95%  confidence  interval  of  1.00  to 
1.04  and  R-Squared  of  0.99. 


Figure  8.  Photo  of  Superimposed  Stereo  Pairs  Used  to  Assess  Pixel  Disparity 
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Stereo  Calibration 
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Figure  9.  Measured  Pixel  Shifts  for  Modeled  Rod  Distances 


Stereo  Calibration 


Figure  10.  Virtual  Model  Rod  Distance  Plotted  Against  Stereo  Displayed  Distance 
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3.4  Display  Class  &  Type:  FoLD  &  Temporally  Scanned  Volume  (TSV) 


A  temporally  scanned  volume  (TSV)  display  produces  a  full-parallax  image  within  a  defined  3D 
space  that  is  viewable  with  the  unaided  eye.  The  display  volume  is  divided  into  addressable 
volume  elements  (voxels).  Ideally,  each  voxel  emits  light  into  all  directions  (4n  sr) 
representative  of  the  corresponding  element  of  a  translucent  3D  model.  The  TSV  type  belongs  to 
the  FoLD  class  of  3D  displays. 

Volumetric  systems  have  been  built  in  a  variety  of  design  approaches  based  on  then-available 
materials  and  components  including:  (a)  emissive  voxel  elements,  such  as  moving  light  emitting 
diode  arrays  (see  e.g.  Ito  et  al.,  2007);  (b)  multiplexed  2D  images,  such  as  the  LIS  Depth  Cube 
zl024  and  xl403a  comprising  a  sequentially  projected  stack  of  some  20  parallel  liquid  crystal 
diffusing  planes  (Osmanis,  2016,  www.lightspace3D.com);  (c)  direct  write  green  laser  beam 
modulated  as  scanned  from  above  (or  below)  onto  a  spinning  helical  screen  (Psaltis,  1992,  1994, 
1996);  (d)  direct  freeform  volumetric  line  writing  by  scanning  the  intersection  point  of  two 
infrared  lasers  in  a  two-photon  up-conversion  medium  (Downing  et  al.,  1996);  and  (e)  projection 
of  2D  slices  of  a  3D  model  onto  a  transflective  moving  screen,  as  in  the  AIS  Perspecta  (Donelan, 
2010).  In  reality,  each  voxel  is  blocked  in  some  directions  by  mechanical  structure  design 
features  of  each  of  these  TSV-type  display  systems. 

All  TSV-type  3D  display  systems  involve  presentation  of  a  complete  3D  frame  of  the  voxel  array 
over  time  and  rely  on  slow  retinal  chemistry  to  achieve  the  human  perception  of  a  complete 
image.  This  temporal  characteristic  is  familiar  from  scanned  2D  display  types,  including  the 
cathode  ray  tube  (CRT).  All  volumetric  systems  have  temporal  and  spatial  visual  artifacts  due  to 
the  limitations  of  the  component  devices  available  when  they  were  designed:  graphic  and  logic 
processors,  optics,  light  emitting  and  modulating  devices,  signal  compression  and 
interconnections,  and  mechanical  movement.  For  example,  both  the  LIS  Depth  Cube  and  AIS 
Perspecta  leverage,  but  are  performance-limited  by,  the  2D  display  DLP  kits  released  by  TI  to 
third-party  developers;  the  2D  DLP  image  resolution  is  far  too  low  to  support  high  definition 
volumetric  display  systems. 

Some  specific  challenges  for  the  AIS  Perspecta  swept-volume  approach  include:  (a)  large  screen 
acceleration  forces;  (b)  synchronization;  (c)  dead  zone  at  the  screen  spin  axis;  (d)  inability  to 
generate  opaque  surfaces  (Brundell  et  al.  1993);  and  (e)  ambient  light  reflections  from  the  white 
screen  (contrast  reduction)  and  from  the  two  clear  concentric  domes  over  the  spinning  screen 
(safety  feature).  However,  for  some  datasets,  translucency  is  a  desirable  feature  and  in  some 
applications  the  room  lights  can  be  sufficiently  dimmed  to  avoid  ambient  light  reflections.  The 
imaging  quality  available  on  the  AIS  Perspecta  is  illustrated  in  Figure  11. 

3.4.1  Rendering  for  Rotating  Screen  TSV 

3D  models  may  be  approximated  as  a  series  of  planes  sampled  in  either  a  Cartesian  or  spherical 
coordinate  (e.g.  parallel  planes  along  the  z-axis  going  away  from  viewer,  or  intersecting  planes 
along  the  (p-axis  spinning  about  the  vertical  y-axis).  Each  sample  plane,  or  slice,  of  the  dynamic 
3D  model  comprises  a  set  of  voxels  that  can  then  be  rendered  as  a  bitmap  for  projection  at  the 
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appropriate  moment  in  time.  The  number  of  bitmaps  can  reach  into  the  hundreds  creating  a 
significant  computational  burden  that  must  all  be  accomplished  within  33  ms  for  a  system  frame 
rate  of  30  Hz.  The  3D-viewpoint  is  selected  by  self-motion  of  the  viewer  with  respect  to  the 
display.  Thus,  this  approach  is  not  dependent  on  a  single-viewpoint  virtual  camera  capturing  a 
view  from  within  the  model  (virtual)  space. 

3.4.2  Display  Under  Evaluation:  Actuality  Integral  Slice  (AIS)  Perspecta 

The  AIS  Perspecta  was  a  low  volume  commercial  display  that  was  produced  roughly  between 
2002  and  2007  (Donelan,  2010).  The  display  evaluated  in  this  study  was  purchased  by  AFRL  to 
support  3D  research  in  2005Q1.  The  AIS  Perspecta  has  a  two-sided  screen  that  has  equal  front 
surface  diffuse  reflection  and  back-surface  diffuse  transmission  characteristics.  The  screen 
rotates  on  a  vertical  spin  axis  at  900  rpm  to  producing  a  30  Hz  volumetric  frame  rate.  Some  198 
planar  slices  of  the  model,  evenly  spaced  from  0-360°  in  spin-angle  (p,  are  projected  at  768x768 
pixel  resolution  onto  the  screen  from  a  Texas  Instruments  1024x768  pixel  Digital  Light 
Projection  (DLP)  engine  during  each  revolution.  The  image  processing  and  projection  rate 
required  is  2970  slices/s  =  1.75  Gpx/s  =  42  Gb/s.  The  diffusely  reflecting  characteristic  of  the 
screen,  and  the  specular  reflections  from  the  two  concentric  protective  spheres  surrounding  the 
spinning  screen,  make  the  AIS  Perspecta  vulnerable  to  significant  image  degradation  due  to 
reflections  of  ambient  illumination. 

3.4.3  Calibration  of  the  AIS  Perspecta 

A  low  power,  red  laser  was  fixed  perpendicular,  on  a  sliding  optical  mount,  to  a  scaled  optical 
rail  as  pictured  in  Figure  12.  Distances,  parallel  to  the  optical  rail,  could  be  measured  inside  the 
display  volume  by  simple  geometric  relationships.  First  the  laser  was  positioned  on  the  rail  so 
that  the  laser  intersected  the  point  of  origin  for  the  distance  to  be  measured  in  the  display.  The 
starting  position  for  the  laser  on  the  rail  was  recorded  and  then  the  laser  was  positioned  so  that  it 
intersected  the  point  of  termination  for  the  distance  to  be  measured.  Again,  the  position  of  the 
laser  on  the  optical  rail  was  recorded.  Since  the  opposite  sides  of  a  parallelogram  are  equal,  the 
distance  between  the  starting  and  ending  position  of  the  laser  on  the  optical  rail  is  equal  to  the 
distance  between  the  points  of  origin  and  termination  in  the  display  feature  (rod  in  this  case). 

For  example,  the  width  of  a  rod  could  be  measured  by  sliding  the  laser  along  the  scale  until  it  is 
pointing  at  the  right  edge  of  the  rod,  then  the  laser  is  moved  along  the  scale  until  it  is  pointing  at 
the  left  edge  of  the  rod.  The  rod  width  is  then  equal  to  the  difference  between  the  laser  locations, 
for  the  right  and  left  alignments.  The  AIS  Pespecta  was  commanded  to  produce  the  white  James 
rods  at  a  variety  of  locations  in  the  volume  addressed  by  the  display.  Measurements  of  the  rods 
were  taken  from  the  viewing  direction  of  the  observer  to  insure  that  the  rods  were  the  correct  size 
and  separation.  Then  the  laser  and  rail  was  rotated  90°  around  the  display  center.  Multiple 
measurements  were  taken  with  rods  at  different  depth  locations  to  confirm  that  stimulus  location 
commands  resulted  in  the  expected  stimulus  movements. 
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Figure  11.  Aircraft  Model  Displayed  in  2D  on  Flat  Panel  (left)  vs.  in  3D  on  AIS  Perspecta  (right) 
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Figure  12.  Remote  Measurement  of  Rod  Images  on  AIS  Perspecta  Using  Orthogonal  Laser  Translation 
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3.5  Display  Class  &  Type:  FoLD  &  Spatially  Integrated  Ray  (SIR) 


An  ideal  electronically-generated  3D  display  would  produce  an  image  that  is  equivalent  to  real 
objects  viewed  in  the  real  world.  The  concept  of  lightfield  has  been  used  to  design  capture 
techniques  to  image  real  world  scenes  in  x-y-z  space  (e.g.  scien.stanford.edu,  accessed  24  Jul 
2016).  Traditional  2D  cameras  capture  an  image  x-y  focused  at  a  single  depth  z.  A  2D  image 
may  be  approximated  as  a  discrete  array  of  rays  originating  at  a  set  of  adjacent  solid  angle 
samples  (i,j),  i=l-Nx,  j=l-Ny.  Similarly,  3D  image  may  be  approximated  as  a  discrete  set  of  rays 
originating  at  a  set  of  adjacent  solid  volume  elements  (i,j,k),  i=l-Nx,  j=l-Ny  k=l-Nk. 

Display  systems  of  the  FoLD  class,  SIR  type  are  designed  invert  the  lightfield  capture  process. 
The  design  objective  is  to  generate  an  approximation  of  the  lightfield  that  a  3D  model  would 
generate  if  it  were  a  real  object  imaged  with  a  lightfield  camera  in  x-y-z  space.  Common 
approaches  generate  perspective  views  comprising  h  x  v  aperiodic  sets  of  spatial  rays  (aka  pencil 
beams).  The  spatial  rays  comprising  a  view  originate  at  non-acljacent  view  pixels  within  an 
extremely  large  N  x  M  pixel  array  grouped  into  n  x  m  subarrays,  where  h  =  N/n  and  v  =  M/m. 
Each  n  x  m  pixel  subarray  together  with  its  associated  microoptic  element  forms  the  physical 
realization  of  a  3D  model  facet  projector  (aka  hogel)  schematically  illustrated  in  Figure  13. 3 
Each  hogel  directs  n’  x  m’  rays  of  light,  where  n’  <  n  and  m’<  n,  into  n’  x  m’  angular  directions 
(0n’  <p  m’  );  each  ray  represents  how  one  3D  model  point  would  appear  in  each  angular  direction. 

The  pupil  (of  an  eye  or  camera)  located  a  particular  position  in  the  design  viewing  zone,  (0,  <1>, 
dm™,  dmax),  captures  about  h  x  v  irregularly  spaced  rays  originating  from  non-adjacent  pixels, 
with  typically  just  zero,  one,  or  two  rays  from  each  hogel.  The  pupil  of  the  other  eye,  located  at 
an  average  human  population  interpupillary  distance  of  65  mm  away,  captures  a  different  set  of  h 
x  v  irregularly  spaced  rays  from  a  different  hogel  set  that  generate  the  image  viewable  from  that 
pupil  position.  The  perceived  image  detail  at  each  pupil  perspective  location  is  approximately  h 
x  v.  All  rays  are  simultaneously  present  whether  a  person  has  positioned  their  head  (or  a  camera 
aperture)  there  or  not.  Thus,  the  correct  3D  perspective  view  is  presented  to  either  eye  of  one  or 
more  persons  naturally  as  they  move  their  heads  throughout  the  field-of-display.  Challenges 
include:  computation  of  pixel  drive  signals  from  the  3D  model;  construction  of  large  array  of  2D 
pixels  by  tiling  microdisplays;  fabrication  of  high  quality  miniature  projection  optic  elements  and 
accurate  registration  over  the  pixel  plane,  total  space-weight -power,  and  acceptable  image  update 
rate,  control  interface,  form  factor  and  noise  profile. 

Ideally,  a  SIR  display  would  have  an  imperceptibly  small  hogel  pitch,  with  each  one  producing  a 
sufficient  number  of  beams  to  fill  the  display  volume  at  an  average  angular  pitch  less  than  that 
subtended  by  the  average  pupil  size  at  the  maximum  design  viewing  distance.  Practically,  each 
hogel  can  only  produce  a  limited  number  of  beams  and  some  viewing  positions  (solid  angle 
direction  and  distances)  may  fall  into  the  space  between  two  rays  and  no  light  from  any  hogel 
enters  the  eye  for  that  model  point.  These  hogel  drop-outs  in  the  projected  field  of  light  rays 


3  The  term  holographic  element  (hogel)  has  been  used  for  some  20  years  (Benton  and  Bove,  2008).  The  word 
holographic  would  be  apt  provided  the  size  of  the  extremely  large  pixel  array  N  x  M  were  to  be  on  the  order  of 
1  Tpx  with  ~5  nm  pixel  pitch  with  ~lpm  nanooptic  element  pitch,  and  provided  a  future  generation  supercomputer 
is  capable  of  rendering  model  points  into  hogels  at  60Hz.  Current  systems  underperform  by  a  factor  of  ~1010. 
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vary  with  changes  in  the  viewing  position  (head  motion)  or  the  3D  model  (dynamic  update),  thus 
creating  discontinuities  that  degrade  the  perceived  display  quality.  Light-field  discontinuities  are 
mitigated  by  the  inability  to  completely  collimate  the  beams. 

Practically,  a  perfectly  collimated  rays  are  not  possible  and  a  tolerance  for  optical  errors  and 
diffusion  are  required  as  part  of  the  design  SIR  display  design.  The  degree  of  beam  collimation  is 
determined  by  many  factors  including  pixel  size,  lens  focal  distance,  and  pixel-optics  alignment. 
All  light  beams  have  some  divergence  and  a  non-uniform  beam  profile.  Divergence  may  cause  a 
particular  direction  to  receive  a  mixture  of  energy  from  two  adjacent  beams  resulting  in  an 
intermediate  perception  so  that  beam  divergence  serves  as  an  interpolation  function,  smoothing 
out  the  discontinuity  between  the  designed  ideal  ray  directions.  While  the  smoothing  associated 
with  beam  divergence  may  be  a  benefit  it  comes  at  a  cost:  ray  crosstalk  that  may  be  perceived, 
for  example,  as  non-uniform  coloration  and  reduced  contrast. 

Example  FoLD  SIR  System:  The  ZIR  9-Tile  ZMD  has  a  large  23040  x  18,432  (424,673,280  px) 
pixel  array  comprising  100  x  100  px  subarrays,  within  which  the  center  76  x  76  px  patch 
generate  5776  rays  representing  a  single  model  point  as  viewable  from  5,776  directions  as 
illustrated  in  Figure  13.  The  total  number  of  addressable  pixels  (=  rays  generated)  is  245  Mpx 


Concept  Underlying  SIR  Display  Design 

Computer  Generated  Hologram  Element  (Hogel) 

A  Hogel  Comprises  an  Array  of  View  Pixels,  Each  Separately  Modulated  in  Intensity  and  Angle 


Hogel 


ZIR  Example: 

Static  Angles,  5776  View  Pixels 


Hogel  Array 
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High  Definition  1920  x  1080  px  (2.1  Mpx)  Microdisplay 
Partitioned  by  a  21  x  12  Array  of  Miniature  Projection  Lenses 
into  252  Hogels 

Note:  Optics  Layers  that  Collect,  Magnify,  and  Collomate  Light  from  View  Pixel  Array  are  not  Shown 

76  x  76  View  Pixel  Array  Each  ZIR  Hogel  Projects  One  3D  Model  Facet  into  76x76  (5776)  Directions 

Figure  13.  Schematic  of  a  Hogel  within  a  Spatially  Integrated  Ray  (SIR)  Display 
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(230.4x184.32x0.76x0.76  px).  The  ZMD  processing  system  must  compute,  from  a  3D  model, 
the  correct  values  with  which  to  address  each  of  245  Mpx  to  produce  245  Mrays  projected  into 
the  viewing  zone.  Each  ~5mm  pupil  (human  eye  or  camera)  located  within  the  design  viewing 
zone  (90  x  90°  pyramidal  field  and  0.5-1  m  distance  from  the  pixel  plane)  collects  a  maximum  of 
230  x  184  (42,320)  irregularly  spaced  rays.  This  image  resolution  which  equates  to  a  so-called 
equivalent  2D  display  resolution  that  is  less  than  the  early  qVGA  (320x240  pixel)  digital  video 
display  standard.  However,  this  230  x  184  perceived  image  resolution  is  projected  in  full 
parallax  simultaneously  across  the  90  x  90°  pyramidal  field-of-view  at  a  frame  rate  of  1-10  Hz. 
The  ZIR  9-Tile  ZMD  system  includes  a  processing  system  comprising  27  circa  2008  networked, 
custom-packaged  workstations  (CPU,  GPU,  FPGA,  HD)  drawing  1  KW.  The  workstations 
receive  are  controlled  by  a  client  PC  running  the  3D  model  and  a  custom  SIR  operating  system 
called  Argon  that  Zebra  Imaging  Inc.  developed  for  its  ZMD  system  (Klug  et  al,  2013). 

Photographs  of  ray  bundles  (lightfield  approximations)  generated  from  a  3D  model  and  rendered 
into  (a)  a  monochrome  photopolymer  hologram  (1  mm  hogel  pitch,  512  x  512  rays/hogel)  and 
(b)  the  ZIR  9-Tile  ZMD  SIR  system  (1.64  hogel  pitch,  76  x  76  rays/hogel)  are  shown  from  the 
same  perspective  for  comparison  in  Figure  14.  The  lower  quality  of  the  image  from  the 
electronic  pixelated  ZIR  system  relative  to  the  hologram  is  obvious.  The  effective  hogel  pitch 
and  non-uniformity  are  much  less  in  the  hologram  than  the  ZIR.  Fight  from  an  object  not  only 
has  a  direction  and  intensity  but  it  also  forms  of  a  wavefront.  It  is  the  curvature  of  the  wavefront 
(from  the  hologram)  that  determines  the  optical  distance  to  the  origin  of  the  beam.  A  beam  with 
a  uniform  wavefront  can  be  well-focused  by  the  eye  according  to  its  point  spread  function  (PSF). 
Therefore,  for  a  perfectly  collimated  display  and  a  well-focused  eye,  the  size  of  the  PSF  relative 
to  the  hogel  pitch  determines  whether  the  image  is  perceived  as  smooth  and  continuous  or  as  a 
discrete  collection  of  spots.  Taking  the  limitations  of  hogel  optics  into  consideration,  an 
imperfect  light  ray  field  is,  in  fact,  produced  comprising  a  mixture  of  wavefront 
directions/curvatures  and  corresponding  optical  distances.  Consequently,  the  viewer  may  not 
focus  the  light  optimally,  resulting  perception  of  a  blurred  image.  Defocus  is  a  well-known 
method  of  low  pass  filtering  of  coarsely  sampled  images  and  may  benefit  SIR  displays  like  ZIR. 

3.5.1  Rendering  for  a  SIR  Display 

One  logical  approach  to  rendering  a  3D  model  is  to  create  a  virtual  camera  for  at  the  center  of 
each  hogel  with  a  virtual  pixel  corresponding  to  each  beam  direction.  This  approach  creates  a 
virtual  miniature  camera  array  that  parallels  the  physical  hogel  miniature  projector  array  and  is 
analogous  to  the  relationship  between  integral  cameras  and  integral  projection.  However,  the 
hogel  miniature  virtual  cameras  do  not  have  the  same  constraints  as  integral  cameras.  For 
example,  an  integral  camera  can  only  record  the  reflections  in  focus  off  of  physical  surfaces  in 
front  of  the  apertures  and  a  set  focus  distance.  In  contrast,  virtual  cameras  can  select  any  surface 
to  record,  first,  last  or  any  surface  in-between.  Therefore  the  3D  model  being  rendered  can  be  in 
front,  behind  or  straddle  the  virtual  camera  array.  Integral  cameras  record  the  reflection  off  the 
first  surface  regardless  of  distance;  virtual  camera  views  can  be  truncated  at  any  distance  and 
angle.  This  truncation  of  the  viewing  cone  results  in  a  frustum-shaped  FoV.  If  the  camera  view 
is  truncated  both  in  front  of  and  behind  the  camera  array,  the  camera  FoV  can  be  described  as 
having  a  double  frustum  shape.  The  computational  demands  for  lightfield  rendering  are  huge. 
Rendering  rates  sufficient  for  interaction  and  smooth  motion  video  are  difficult  to  achieve. 
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Figure  14.  Photographs  of  Lightfields  Produced  from  a  3D  Model  Written  into  a  Hologram  (left)  and  ZIR  9-Tile  ZMD  (right) 
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3.5.2  Device  Under  Test:  Zebra  Integral  Ray  (ZIR)  1-Tile  Zscape  Motion  Display  (ZMD) 

The  light-field  display  used  in  this  effort  was  the  mobile,  1-Tile  version  of  the  ZIR  9-Tile  ZMD 
developed  by  Zebra  Imaging  Inc.  of  Austin  TX  under  the  DARPA  Urban  Photonic  Sandtable 
Display  program.  The  ZIR  1-Tile  ZMD  comprises  a  rectilinear  84  x  72  array  of  hogels  (hogels) 
that  project  ray  bundles  representing  each  of  6,048  3D  model  points.  Each  1.64  mm  diameter 
hogel  projects  quasi-collimated  rays  whose  intensity  is  individually  modulated  into  5,776  fixed 
directions,  as  illustrated  in  Figure  13.  These  rays  span  a  90x90°  (2.1  sr)  square  pyramidal 
viewing  zone  whose  axis  is  perpendicular  to  the  planar  hogel  array.  The  hogels  were 
implemented  in  a  so-called  ZIR  ZMD  “Tile”  comprising  (a)  a  4x6  array  of  Seiko-Epson  high 
definition  1920  x  1080  px  (2.1Mpx)  transmissive  active  matrix  liquid  crystal  microdisplays, 
covered  by  (b)  a  miniature  projection  optics  lens  array  and  driven  from  beneath  by  (c)  drive 
electronics.  The  Tile-sized  miniature  projection  lens  array  was  overlaid  on  the  4x6  microdisplay 
array  with  an  optical  pitch  and  alignment  such  that  a  21  x  12  (252)  miniature  project  lens 
subarray  overlaid  each  microdisplay,  completing  a  corresponding  21  x  12  subarray  of  hogels 
(Klug  et  al.,  2013). 

The  lightfield  is  low  pass  filtered  by  a  diffusing  screen  placed  about  25  mm  above  the  top  of  the 
miniature  projection  lens  layer.  The  ZIR  image  processing  system  ingests  OpenGL  scene 
descriptions  (shunted  by  the  Zebra  Argon  operating  system  from  the  client  PC  to  the  ZMD)  and 
distributes  the  data  to  a  three-slice  parallel  computing  engine.  Each  Tile  comprises  three  so- 
called  “Slices,  each  comprising  a  workstation  driving  a  4x2  subarray  of  the  4x6  microdisplay 
array.  Each  workstation  is  housed  within  a  30  x  6  x  6  in.  frame  containing  a  component  stack 
comprising  a  20  GB  hard  drive,  a  CPU  motherboard,  GPU  daughterboard,  and  an  FPGA  card 
programmed  to  generate  the  drive  signals  required  by  the  microdisplays.  This  rendering  engine 
implemented  a  double  square  frustum  camera  modeled  on  the  square  pyramidal  emission  zone 
of  the  display.  Image  update  rate  is  better  than  4  Hz  for  simple,  but  <  1  Hz  for  complex,  3D 
models.  The  microdisplays  can  be  operated  in  color  sequential  mode  with  the  built-in  RGB 
light-emitting  diode  (LED)  backlighting  system  to  generate  some  4000  colors.  The  maximum 
luminance  is  200  cd/m2  with  a  70:1  contrast  ratio  under  a  dim  room  illumination  condition. 

The  ZIR  1-Tile  ZMD  system  can  be  scaled  to  form  larger  display  systems  if  desired,  such  as  the 
3x3  Tile  (ZIR  9-Tile  ZMD)  system  described  above. 

3.5.3  Calibration  of  the  ZIR  1-Tile  ZMD 

Initial  calibration  of  the  ZMD  was  accomplished  by  Zebra  Imaging  Inc.,  but  visual  inspection  of 
the  James  two-rod  stimulus  implementation  on  the  ZMD  suggested  that  the  display  had  a 
noticeable  tilt  to  objects  coming  up  out  of  the  display  surface.  Attempts  at  AFRL  to  recalibrate 
the  display  using  the  Zebra  automated  procedure  proved  difficult  and  produced  large  patches  of 
non-aligned  hogels.  The  Zebra  calibration  tool  allowed  for  the  manual  adjustment  of  individual 
hogels,  but  was  labor  intensive.  An  alternative  method  of  calibration  was  developed  at  AFRL  by 
one  author  (Heft)  that  allowed  for  optimum  display  uniformity  and  image  quality.  Calibration  of 
the  display  via  the  Heft  method  is  a  two-step  process.  First,  an  intra-hogel  alignment  of  the  hogel 
optic  and  the  hogel  pixel  array  is  accomplished  to  optimize  imaging  performance.  Then  scaling 
factors  were  calculated  for  the  OpenGL  coordinate  system  to  insure  that  the  ZMD  display 
volume  was  uniform  and  accurately  scaled  for  displaying  the  James  rods. 
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3.5.3.1  Intra-Hogel  Alignment  of  the  ZIR  1-Tile  ZMD.  After  the  micro-lens  array  is  overlaid 
on  the  micro-display,  thereby  fixing  the  hogel  locations,  the  optimal  76  x  76  pixel  array  from  the 
microdisplay  must  be  assigned  and  aligned  to  each  hogel.  The  best  pixel  array  is  determined  by 
the  intercept  of  the  optic  axis  of  the  microlens  with  the  microdisplay  surface.  The  intercept  also 
serves  as  the  origin  of  the  coordinate  system  from  which  pixel  position  and  therefore  beam 
direction  can  be  estimated.  Determining  the  optical  axis  intercept  was  an  iterative  process.  A 
small  cluster  of  pixels  is  illuminated  under  each  hogel  micro-lens.  The  centroid  of  the  cluster 
functioned  as  a  cursor  which  could  be  moved.  A  high  resolution  camera  is  positioned  6  ft  from 
the  display  surface  (optimal  camera  distance  is  dependent  on  ZMD  size  and  camera  resolution) 
with  its  optic  axis  perpendicular  to  the  center  of  the  ZMD  surface.  Using  this  cursor/camera 
combination,  intensity  as  a  function  of  location  is  recorded.  For  the  hogels  in  the  center  of  the 
ZMD,  directly  in  front  of  the  camera,  the  intensity  function  is  unimodal  with  the  maximum 
intensity  recorded  when  the  cursor  and  micro-lens  optical  axis  are  coincident.  For  non-center 
hogels,  the  intensity  function  was  also  unimodal,  but — due  to  the  slightly  oblique  camera  view — 
the  maximum  intensity  occurs  at  a  slight  misalignment  of  the  axes  of  the  curser  and  micro-lens 
optic.  Therefore,  the  cursor  location  at  maximum  recorded  intensity  is  taken  to  be  the  best 
estimate  of  the  micro-lens  optic  axis  and  the  center  of  the  optimal  pixel  array. 

In  the  manual  Zebra  Imaging  calibration  procedure,  the  movement  of  the  cursor  is  made  using 
key  strokes  and  mouse  inputs.  This  manual  procedure  is  automated  here  using  OpenCV  and 
AutoHotKey  to  emulate  the  manual  inputs.  Figure  15  shows  a  series  of  images  taken  while 
OpenCV  and  AutoHotKey  inputs  drove  the  cursor  to  the  maximum  intensity  location. 

The  automated  iteration  process  took  many  hours  due  to  the  large  number  of  hogels  involved. 
Figure  16  shows  the  original  appearance  of  the  cursors  (calibration  pattern)  side-by-side  with  the 
test  pattern  after  the  top  two-thirds  of  the  hogels  had  been  aligned.  Figure  17  shows  the  test 
pattern  after  alignment.  The  residual  variation  in  cursor  intensity  is  disappointing,  but  overall  the 
imaging  performance  of  the  display  is  substantially  improved.  After  the  alignment  is  complete 
and  the  center  of  the  pixel  array  determined,  the  individual  pixels  can  be  mapped  to  a  beam 
direction  using  the  pixel  location  and  a  standard  hogel  beam  distribution  model. 
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Figure  15.  OpenCV  and  AutoHotKey  Inputs  Move  Cursor  to  Maximum  Intensity  Location 
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Figure  16.  Hogel  Calibration  Pattern:  Initial  (Left)  and  Top  Two-Thirds  Realigned  (Right) 
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Figure  17.  Calibration  Pattern  Appearance  after  Alignment 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited.  88ABW  Cleared  09/08/2016;  88ABW-2016-4404. 


3.5.3.2  Display  Volume  Uniformity  of  the  ZIR  9-Tile  ZMD.  To  accurately  reproduce  the 
James  rods,  the  OpenGL  Euclidean  space  has  to  be  sized  and  mapped  uniformly  to  the  ZMD 
display  volume.  To  accomplish  this  mapping,  scaling  factors  are  calculated  for  each  of  the  three 
primary,  orthogonal  dimensions:  width,  height  and  depth.  The  width  and  height  dimensions  are 
defined  as  being  in  the  plane  of  the  hogel  array;  and  depth,  perpendicular  to  this  plane.  Once 
calculated,  there  are  multiple  locations  in  the  rendering  path  where  scaling  factors  could  be 
implemented.  Scaling  factors  in  width  and  height  are  easily  calculated  from  measurements  at  the 
plane  of  the  hogel  array.  An  OpenGL  input  used  to  move  an  object  in  either  width  or  height 
direction,  and  then  the  size  of  the  movements  are  measured  with  a  millimeter  rule.  Once  the 
initial  input-response  ratio  is  determined,  a  simple  correction  factor  can  be  calculated  to  scale  the 
response  to  the  desired  size. 

The  calibration  of  depth  can  be  conducted  in  a  similar  manner  to  the  width  and  height  except  for 
one  significant  complication.  The  display  produces  imagery  both  in  front  of  (real  imagery)  and 
behind  (virtual  imagery)  the  light  emitter  array  surface.  Items  projected  in  front  of  the  surface 
form  real  images,  whose  locations  can  be  confirmed  with  a  scale  or  gauge.  However,  images 
projected  as  if  originating  from  behind  the  display  surface  are  virtual  and,  thus,  inaccessible  to 
direct  measurement. 

Indirect  measurement  is  accomplished  as  follows.  The  entire  image  volume  (real  and  virtual 
portions)  are  reimaged  into  real  image  space  using  a  spherical  lens  of  known  power  mounted  on 
an  indexed  optical  rail  as  shown  in  Figure  18.  A  translucent  diffusing  screen  is  used  to  identify 
the  reimaged  (2nd  image)  locations  produced  by  the  simple  optical  system.  Using  the  lens 
power,  lens  location  and  the  secondary  image  locations  initial  displayed  image  locations  relative 
to  the  lens  can  be  calculated  using  the  paraxial  formula  for  thin  lenses  in  air.  A  three  bar  test 
pattern  was  presented  at  a  series  of  nominal  command  depths  in  OpenGL  space: 

(-0.20,  -0.15,-0.10,-0.05,  0.00,  +0.05,  +0.10,  +0.15,  +0.20).  The  OpenGL  command  depth 
“0.00”  is  assumed  to  correspond  to  the  display  surface  (top  of  the  miniature  projection  optics 
layer)  but  no  other  scaling  or  position  information  is  known  beforehand.  The  series  of  test 
patterns  at  the  nominal  OpenGL  command  depths  are  reimaged  by  the  optical  system  and  the 
secondary  image  locations  are  found  by  adjusting  the  screen  distance  from  the  lens  to  find  the 
position  of  best  image  focus.  From  that  measurement  the  initial  image  positions  relative  to  the 
lens  are  calculated. 

The  subjective  nature  of  determining  the  image  location  by  best  focus  is  compensated  by 
repeating  the  measurement  series  using  several  different  observers  with  each  observer  collecting 
three  complete  sets  of  data.  For  the  present  work  three  observers  are  used  to  take  a  total  of  nine 
sets  of  measurements;  the  cumulative  data  is  plotted  and  fitted  with  a  linear  trend  line.  The 
resulting  estimated  position  relative  to  the  lens  for  command  depth  “0.00”  (display  surface)  was 
within  1  mm  of  the  measured  position  of  the  display  surface.  The  data  is  then  translated  to 
report  all  command  positions  relative  to  the  display  surface.  The  translated  data  is  presented  in 
Figure  19.  The  slope  of  the  linear  regression  line  shown  in  Figure  19  serves  as  the  initial  input- 
response  ratio  from  which  the  correction  factor  can  be  calculated. 
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Diffusing  Screen 


Converging  Lens 
power  =  7.9D 
50  mm  diam 

327  mm  above  display  surface 


Figure  18.  Optical  Set-Up  for  Estimating  Displayed  Image  Location  from  Secondary  Image 
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3.6  Objective  Human  Subject  Evaluation  of  Proposed  ej  Methodology 


The  four  true  3D  devices  described  above  were  arranged  side-by-side  for  efficient  sequential 
objective  evaluation  of  the  proposed  eJ  methodology  by  six  (6)  human  subjects.  The  number  of 
subjects  is  sufficient  to  provide  an  indication  of  the  usefulness,  or  lack  thereof,  of  the  proposed 
methodology,  but  insufficient  for  statistically  significant  conclusions. 

3.6.1  Experimental  Condition  for  ej  Evaluation 

The  experimental  condition  for  the  collection  of  objective  human  subject  evaluation  data  on  the 
four  3D  devices  is  illustrated  in  Figures  20  and  21.  The  subject  is  seated  6  m  from  and, 
sequentially,  directly  in  front  of  one  of  the  four  devices  under  test  (the  other  three  turned  off) 
with  a  hand  controller  to  input  judgments  of  distance. 

3.6.2  Testing  Approaches 

Two  objective  human  subject  testing  techniques  are  common  and  were  both  employed  in  the 
presently  reported  eJ  experiments:  the  Method  of  Adjustment  and  the  Method  of  Alternatives. 
The  viewing  distance  was  6  m  (20  feet)  to  correspond  to  the  normal  testing  distance  for  the  DPA. 


Figure  20.  Close-up  of  Four  3D  Devices  Under  Test  as  Arraigned  for  Subject  Evaluations 


40 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-20 16-4404. 


Figure  21.  Experimental  Condition  for  Human  Subject  Evaluation  of  ej  Methodology 
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3.6.2.1  Method  of  Adjustment  (Subject  Moves  Right  Rod  to  Match  Depth  of  Left).  The 

well-known  psycho-physical  testing  Method  of  Adjustment  (MoA)  was  used  by  Deyo  (1922): 
the  stimulus  depth  magnitude  of  the  right  rod  was  under  the  manual  control  of  the  subject.  The 
subject  in  the  present  work  was  also  required  to  move  the  position  of  a  control  rod  (always  on 
right)  to  match  the  depth  positioning  of  a  fixed  target  rod  (on  left).  The  movement  of  the  control 
rod  was  conducted  via  a  wireless  hand-held  gamepad  controller,  using  the  forward/backward 
directional  thumb-stick.  On  each  trial,  the  participant  had  unlimited  time  to  complete  their 
alignment. 

Each  participant  completed  a  total  of  60  trials  per  display  plus  12  trials  on  the  physical  DPA.  The 
left  rod  was  always  the  fixed  reference  target,  and  the  right  rod  was  always  the  control  object 
whose  position  was  under  manual  control  of  the  viewer.  For  the  electronic  3D  displays,  the  left 
target  rod  could  appear  at  any  one  of  the  five  possible  locations  in  depth  (594,  597,  600,  603,  and 
606  cm)  while  the  right  rod  would  appear  at  a  randomized  starting  location  within  594  to  606 
cm.  With  12  repetitions  per  target  distance  per  display,  this  gave  a  total  of  72  trials  per 
participant.  Data  collection  lasted  approximately  15  minutes  per  display  condition,  so  a  total  of 
about  1  hour  per  participant. 

3.6.2.2  Method  of  Constant  Stimuli  (Forced  Choice:  Subject  Identifies  the  Nearer  Rod). 

The  well-known  psychophysical  Method  of  Constant  Stimuli  (MoCS)  was  used  by  James  (1907, 
1908)  and  Howard  (1919a,  1920a).  The  stimuli  depth  magnitudes  were  chosen  in  advance  at 
random  and  were  not  under  the  control  of  the  subject.  This  testing  procedure  is  also  known  as  the 
Method  of  Alternatives  and,  in  this  case,  as  Two- Alternative  Forced  Choice  (2AFC).  The  stimuli 
was  suddenly  revealed/flashed-on  and  the  subject  was  merely  required  to  make  a  forced-choice 
judgment  about  whether  the  left  or  right  rod  was  perceived  to  be  nearest  (one  was  always  closer 
than  another).  The  answer  was  signaled  with  a  button  press,  using  a  wireless  hand-held  gamepad 
controller.  On  each  trial,  the  participant  had  unlimited  time  to  complete  their  judgment. 

Each  participant  completed  a  total  of  200  trials  on  each  display,  with  the  stimulus  pairs  presented 
in  a  randomized  order.  Each  stimulus  pair  consisted  of  two  rods  (left  and  right),  with  each  rod  at 
one  of  five  possible  locations  in  depth  (594,  597,  600,  603,  and  606  cm)  but  never  at  the  same 
locations  as  the  other  for  any  given  trial.  All  possible  permutations  of  these  pairs  thus  gave  20 
unique  stimulus  pairs,  with  10  repetitions  of  each  pair  (200  trials  per  participant).  Data  collection 
lasted  approximately  15  minutes  per  display  condition,  so  a  total  of  about  1  hour  per  participant. 

Normally,  using  this  procedure,  a  psychophysical  threshold  is  estimated  by  determining  the 
intensity  or  magnitude  of  a  physical  stimulus  required  for  a  subject  to  achieve  some  targeted 
percent  correct  value.  For  a  MoCS  2AFC  paradigm,  75%  correct  is  often  the  chosen  target  level 
of  performance,  midway  between  50%  which  represents  performance  at  chance  levels,  and  100% 
which  represents  perfect  performance. 

Poor  image  quality  of  some  of  the  electronic  3D  displays  made  the  MoCS  estimation  of  a  depth 
acuity  threshold  difficult. 
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3.6.3  Description  of  the  Subjects  Used  to  Evaluate  the  ej  Methodology 

A  total  of  six  (6)  subjects  were  used  to  evaluate  the  proposed  eJ  methodology.  All  subjects  were 
employees  within  the  AFRL  Warfighter  Interface  Division  and  have  extensive  experience  in 
human  subject  testing  of  visualization  systems. 

3.6.3.1  Subject  Selection  Criteria.  All  participants  were  required  to  possess  20/20  distance 
vision  (Snellen  normal)  or  better  in  each  eye  separately  (monocular),  and  together  (binocular). 
Vision  correction  was  acceptable.  We  also  required  all  participants  to  demonstrate  normal 
binocular  depth  vision,  confirmed  via  Titmus  “Stereo  Fly”  test  (Wirt  circles  portion)  which 
confirmed  stereoacuity  of  40  arcsec  or  better.  There  was  no  color  vision  requirement  for  this 
study. 

3.6.3.2  Anonymous  Demographic  Vectors.  The  subjects  were  all  male  and  aged  30-60  years. 


43 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-2016-4404. 


4.0  RESULTS  AND  DISCUSSION 


4.1  Implementation  of  the  Two-Bar  Stimuli  on  the  Three  Electronic  3D  Displays 

The  software-defined  bars  presented  on  the  three  electronic  3D  display  systems  were  designed  to 
be  10  mm  wide  to  match  the  diameter  of  the  rods  in  the  physical  DPA  as  illustrated  in  Figure  22 
for  the  S3D  system.  The  angle  subtended  by  the  10  mm  rod  at  6.00  m  is  1.67  mrad  (5.74  arcmin). 

Theoretically,  as  a  bar  moves  towards  (away)  from  an  observer,  the  angle  subtended  at  each  of 
their  pupils  increases  (decreases)  and  might  provide  a  non-depth  cue  during  the  collection  of 
data.  For  observers  located  at  6  m  (600  cm)  from  the  reference  plane,  the  angle  changes  are 
slight  and  have  been  shown  to  have  no  significant  effect  on  depth  acuity  measured  with  the 
physical  DPA  (Howard  1919)  (Berry,  Riggs,  and  Richards,  1950).  However,  significant  visual 
artifacts  of  the  electronic  3D  systems  employed  in  the  present  work — low  contrast  due  to 
scattered  light  (see  Figure  20);  ill-defined  edges  due  to  diffusion  and  pixilation  effects  (see 
Figures  22-27) — might  also  provide  non-depth  perception  cues  regarding  relative  bar  positions. 
Thus,  the  software-defined  bars  were  designed  to  change  width  with  their  depth  placement  so  as 
to  maintain  constant  angular  subtend  of  1 .67  mrad  at  the  pupils  of  observers  located  at  6  m  from 
the  reference  plane  of  the  display  under  test. 
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Figure  23.  Ill-defined  Edges  of  Bars  on  S3D  Laptop  Display  Due  to  Pixellation  Effects 


Photograph  arrays  are  presented  in  Figures  24-27  to  document  the  appearance  of  the  rod  pairs  for 
the  physical  apparatus  (AFRL- Automated  USAFSAM  variant  of  the  Howard  DPA)  and  the 
rendered  bar  pairs  on  the  three  electronic  3D  displays. 

Figure  25  is  a  1x5  array  of  photographs  taken  of  the  physical  apparatus  (Howard  DPA)  with 
camera  located  6  m  from  reference  position  of  the  fixed  left  rod  at  600  cm  for  a  series  of  right 
rod  distances  of  606,  603,  600,  597,  and  594  cm. 

Figures  25-27  for  the  electronic  3D  displays  present  5x5  arrays  of  images  of  the  rendered  bar 
pairs  taken  with  the  camera  located  6  m  from  their  reference  positions  (600,  600  cm) 
for  all  combinations  of  0,  ±3,  ±6  cm  changes  in  the  position  of  either  bar. 

The  ill-defined  bar  edges  are  readily  apparent  in  these  photographs,  especially  for  the  spatially 
integral  ray  (ZIR  1-Tile  ZMD)  system  (Figure  27).  Tiling  artifacts  of  the  subcomponents  used 
to  construct  the  ZIR  system  are  also  readily  apparent  in  these  images. 


45 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-20 16-4404. 


Figure  24.  Photograph  Array  of  the  Physical  Depth  Perception  Apparatus  (Howard  DPA) 
Taken  with  Camera  Located  6  m  from  Reference  Position  of  the  Fixed  Left  Rod  at  600  cm 
for  Right  Rod  Distances  (Top  to  Bottom)  of  606,  603,  600,  597,  and  594  cm. 
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Figure  25.  Photograph  Array  of  the  S3D  Display  (Dell  M6700  Laptop)  Bar  Pairs 
Taken  with  Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm) 
for  All  Combinations  of  0,  +3,  ±6  cm  Changes  in  the  Position  of  Either  Bar 
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Figure  26.  Photograph  Array  of  the  Volumetric  (AIS  Perspecta)  Display  Bar  Pairs 
Taken  with  Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm) 
for  All  Combinations  of  0,  +3,  ±6  cm  Changes  in  the  Position  of  Either  Bar 
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Figure  27.  Photograph  Array  of  the  Integral  Ray  (ZIR  1-Tile  ZMD)  Display  Bar  Pairs 
Taken  with  Camera  Located  6  m  from  Their  Reference  Position  (600,  600  cm) 
for  All  Combinations  of  0,  +3,  ±6  cm  Changes  in  the  Position  of  Either  Bar 
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4.2  Two-Bar  Stimuli  Depth  Acuity  Testing  of  Six  Observers  on  the  Four  3D  Devices 


Depth  acuity  datasets  were  collected  and  analyzed  for  six  binocular  observers  using  the  method 
of  adjustment  (MoA)  and  the  method  of  constant  stimuli  (MoCS).  The  MoCS  method  is  also 
known  as  the  method  of  alternatives  and  as  the  method  of  forced  choice.  A  third  dataset  for  five 
observers  using  only  their  dominant  eye  (other  blocked)  was  collected  using  the  MoCS  method 
to  assess  monocular  viewing. 

The  four  devices  presented  four  display  conditions  of  the  two-bar  stimuli  to  the  observers.  These 
devices  were  arrayed  in  front  of  the  observer  (as  shown  in  Figures  1,  20  and  21)  from  their  left  to 
right  as  follows:  AIS  (“Perspecta”),  S3D  (“Laptop”),  ZIR  (“Zebra”),  and  DPA  (“RealHD”). 
The  parenthetical  names  are  used  below  to  label  depth  threshold  data  collected  for  each  device. 
Note:  the  data  panels  in  Figures  28-37  are  NOT  ordered  according  to  the  experimental  condition. 

4.2.1  Dataset  1:  Depth  Acuity  Threshold  Measured  by  Method  of  Adjustment  (MoA) 

The  MoA  data  is  analyzed  in  several  ways  which  are  plotted  in  Figures  28-32.  Data  for  all  six 
observers  are  combined  for  each  device  (aka  display  condition)  into  a  subpanel  of  each  figure. 

Note  that  a  fewer  numbers  of  trials  (12/observer)  were  recorded  for  the  physical  device,  the  DPA 
(RealHD),  due  to  the  fixed  location  of  the  left,  reference  bar  at  the  “0”  position.  The  same 
number  of  trials  (60/observer)  were  used  for  each  of  the  three  electronic  3D  displays.  Thus,  the 
DPA  (“RealHD”)  histogram  contains  fewer  trials  (72)  than  those  for  the  other  devices  (360). 

4.2.1. 1  MoA  Signed  Positional  Error  Data.  Figure  28  presents  histograms  showing  the 
frequencies  of  signed  errors  (placement  accuracy)  centered  on  zero  and  normalized  on  the  range 
-12  to  +12.  A  fitted  normal  Gaussian  curve  is  included  for  comparison  within  each  panel. 
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Figure  28.  Method  of  Adjustment  (MoA)  Frequency  Histograms  of  Signed  Error 
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The  positioning  performance  on  the  DPA  (“RealHD”)  was  very  good  with  high  accuracy:  most 
observations  were  centered  around  zero,  with  perhaps  a  slight  tendency  to  underestimate  the 
depth  position  (tail  towards  the  negative  signed  error  range).  This  result  seems  to  confirm  the 
utility  of  the  physical  DPA  for  depth  perception  testing. 

The  S3D  (“Laptop”)  performance  was  centered  at  zero,  but  with  a  wide  distribution  of  signed 
errors  in  both  the  negative  and  positive  directions.  These  data  suggest  that  this  stereo  display 
generally  supports  depth  perception  but  with  considerably  less  accuracy  and  consistency  than  is 
offered  by  the  DPA  (“RealHD”)  system. 

The  AIS  (“Perspecta”)  and  the  ZIR  (“Zebra”)  devices  had  similar-looking  histograms  comprising 
a  large  spike  in  the  distributions  of  errors  centered  around  zero  and  smaller  but  still-prominent 
spikes  at  -12,  -6,  +6,  and  +12.  These  data  suggest  that  perceptual  confusions  were  systematically 
occurring  at  particular  relative  distances.  Depth-dependent  visual  artifacts  (luminance/contrast, 
and/or  blur)  were  observed  for  both  the  Perspecta  and  the  Zebra;  the  presence  and  intensity  of 
these  artifacts  seemed  to  primarily  depend  on  the  depth  locations  of  stimuli  relative  to  the  zero 
reference  depth  (“center”)  of  the  device.  In  other  words,  these  systematic  error  spikes  in  the 
histograms  likely  indicate  front/back  positional  confusions  caused  by  the  artifacts.  These 
confusions  may  have  impeded  the  ability  observers  to  perform  accurate  depth  alignments  as 
desired,  but  only  at  particular  relative  depths.  This  issue  is  addressed  in  section  4.3.2. 

4.2.1.2  MoA  Absolute  Positional  Error  Data.  Figure  29  presents  histograms  of  absolute 
positional  error  observed  for  the  four  test  devices.  Observations  based  on  this  data  are  similar  to 
those  for  the  raw  signed  error  data  from  which  it  is  derived. 

For  the  DPA,  errors  tended  heavily  towards  zero,  with  almost  no  tail  observations  beyond  5  cm 
of  error,  again  confirming  that  the  real  apparatus  allows  for  an  accurate  perception  of  rod 
locations  in  depth. 

For  the  S3D,  the  shape  of  the  distribution  was  very  similar  to  the  DPA,  though  the  tail  is 
considerably  lengthier,  evidencing  errors  as  large  as  10  to  15  cm  in  some  cases. 

However,  the  systematic  confusions  caused  by  the  circular  rotating  screen  of  the  AIS 
(“Perspecta”)  and  the  low  resolution  hogels  of  the  ZIR  (“Zebra”)  devices  are  even  more 
prominent,  relative  to  the  S3D  laptop  and  the  DPA.  These  two  FoLD  systems  (ZIR,  AIS)  again 
showed  similar  distributions  to  each  other,  with  large  spikes  tending  towards  zero  with 
prominent  error  “bumps”  clustered  at  around  6  and  12  cm.  This  new  look  at  the  raw  signed  error 
data  supports  the  above  “artifact”  interpretation  of  these  error  clusters. 

These  unique  display  quality  problems,  which  are  inherent  in  the  circa  2005  Perspecta  and  circa 
2010  Zebra  devices  used,  may  have  unintentionally  turned  what  was  meant  to  be  a  depth- 
matching  task  into  instead  a  visual-artifact-matching  task',  at  least  for  these  two  FoLD  systems. 
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Figure  29.  Method  of  Adjustment  (MoA)  Frequency  Histograms  of  Absolute  Error 


4.2.1.2.1  Mean  Placement  Performance  as  Function  of  Reference  Distance 

Figure  30  shows  the  placement  performance  (distance  observer  placed  the  moveable  pin)  versus 
the  reference  distance,  averaged  over  all  trials.  Theoretically  optimal  performance  is  indicated  by 
the  solid  red  y=x  hne;  deviations  from  this  line  indicate  suboptimal  performance. 

These  plots  of  average  performance  suggest  that  the  perception  of  depth  location  did  scale  with 
distance,  as  the  slopes  are  positive  and  with  patterns  approaching  linearity,  though  for  the  S3D 
(“Laptop”),  AIS  (“Perspecta”),  and  ZIR  (“Zebra”)  devices  the  error  bars  were  noticeably  larger 
when  compared  to  the  DPA  (“RealHD”)  device. 

The  DPA  clearly  resulted  in  extremely  accurate  performance  with  very  tight  error  bars,  for  its 
single  reference  pin  position. 

Interestingly,  the  S3D  (“Laptop”)  data  indicate  consistently  inaccurate  location  judgments: 
observers  tended  to  misperceive  the  stereo  stimuli  as  being  nearer  than  expected,  except  for  the 
closest  reference  distances. 

The  AIS  (“Perspecta”)  and  ZIR  (“Zebra”)  devices  seemed  to  have  particular  problems  portraying 
nearer  locations.  However,  both  the  AIS  and  ZIR  portrayed  stimuli  at  the  center  (real  or  virtual 
“screen  surface”)  positions  accurately  and  with  low  variability  in  judgments. 
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Figure  30.  Placement  Performance  vs.  Reference  Distance  (Mean  +  1  SEM); 
Straight  Red  Line  Represents  Optimal  Performance 


4.2.1.2.2  Per  Trial  Placement  Performance  as  Function  of  Reference  Distance 

Figure  3 1  is  a  scatterplot  of  all  MoA  data.  Here,  instead  of  looking  at  the  dataset  data  as 
averages  with  error  bars,  every  single  observation,  every  data  point  per  trial,  is  plotted.  The  x 
and  y  variables  in  Figure  31  are  identical  to  those  Figure  30  and  a  red  y=x  line  is  used  to  indicate 
theoretically  perfect  performance.  However,  the  scatterplot  shows  how  noisy  the  relative 
performance  was  across  the  four  devices.  The  DP  A  (“RealHD”)  had  good,  consistent 
performance  in  placement  accuracy,  with  very  little  scatter. 

The  S3D  (“Laptop”)  had  extremely  noisy  data  and  inconsistent  placements.  The  slope  shown  for 
the  averaged  S3D  data  in  Figure  30  is  barely  perceptible  in  the  raw  data  scatterplot  in  Figure  31. 

The  scatterplots  for  the  two  FoLD  systems  AIS  (“Perspecta”)  and  ZIR  (“Zebra”)  demonstrated 
very  similar  patterns:  clusters  of  data  points  were  aligned  accurately  along  the  red  y=x  line 
(perfect  performance)  with  shockingly  remarkable  precision. 

However,  the  scatterplots  for  these  two  FoLD  systems  also  resulted  in  clusters  on  an  oppositely- 
sloped  y  =  (-x)  line,  which  crosses  the  optimal  performance  y=x  line  at  the  center  600  cm 
distance.  This  y  =  (-x)  line  represents  “perfectly  wrong  performance”  and  is  depicted  as  by 
dashed  blue  lines  in  the  AIS  and  ZIR  panels  within  Figure  31.  The  clusters  on  this  y  =  (-x) 
dashed  blue  line  demonstrate  that  front/back  positional  confusions  or  misperceptions  were 
occurring.  For  instance,  reference  target  pins  at  a  distance  of  606  cm  (+6  cm  from  center 
position)  were  sometimes  being  misperceived  at  594  cm  (-6  cm  from  center),  and  vice  versa. 
Likewise,  reference  locations  of  603  cm  (+3)  were  occasionally  but  consistently  being 
misperceived  at  597  cm  (-3),  and  vice  versa.  This  front/back  confusion  was  not  observed  in  the 
DPA  or  S3D  data. 
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Figure  31.  Per  Trial  Placement  Performance  vs.  Reference  Distance;  Solid  Red  Lines  Show 
Optimal  Performance;  Dashed  Blue  Lines  Indicate  Systematic  Near/Far  Misperceptions 


The  front/back  confusion  pattern  found  in  the  MoA  dataset  for  the  AIS  (“Perspecta”)  and  ZIR 
(“Zebra”)  devices  may  be  caused  by  the  aforementioned  visual  artifacts  of  these  two  displays. 
The  magnitude  and  severity  of  the  artifacts  were  observed  to  depend  upon  the  magnitude  of  the 
distance  a  stimulus  was  commanded  to  be  displayed  relative  to  its  “center”  plane.  The  farther 
from  this  central  position  in  either  direction  of  depth  a  stimulus  was  placed,  the  more  degraded 
the  displayed  image  of  the  stimulus  became  (in  terms  of  blur,  intensity  changes,  etc.).  Thus, 
distances  of  ±3  cm  result  in  perceptually-similar  displayed  stimuli  (suffering  from  similar 
magnitudes  of  artifacts).  Distances  of  ±6  cm  resulted  in  even  worse  artifacts  (but  again  of 
similar  magnitudes). 

Analysis  of  the  MoA  dataset  using  only  averages  (similar  to  the  way  psychophysical  stereoacuity 
threshold  data  are  often  computed)  may  be  misleading.  Table  1  summarizes  averaged  MoA  data 
for  the  four  3D  devices  used  in  this  work.  The  mean  absolute  error  for  the  DPA  (“RealHD”) 
device  was  1.32  cm  was  lowest  (best);  AIS  (“Perspecta”),  2.57  cm;  ZIR  (“Zebra”),  3.44  cm;  and 
S3D  (“Laptop”),  4.77  cm.  One  might  be  tempted  to  conclude  that  the  AIS  and  ZIR  are  capable 
of  supporting  depth  stimuli  that  significantly  outperform  that  offered  by  the  stereo  display. 
However,  the  presence  of  the  visual  artifacts  in  the  AIS  and  ZIR  devices  used  in  this  work  may 
make  this  a  premature  and  inaccurate  conclusion. 


54 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-20 16-4404. 


Table  1.  Means  and  Standard  Deviations  of  Absolute  Error  Performance,  across 
Participants,  for  Four  3D  Devices  Using  MoA  Dataset  1 


Device  (Viewing  Condition) 

Mean  (cm) 

Standard  Deviation  (cm) 

S3D  (“Laptop”) 

4.77 

3.76 

AIS  (“Percepta”) 

2.57 

3.50 

DPA  (“RealHD”) 

1.32 

1.24 

ZIR  (“Zebra”) 

3.44 

4.28 

Figure  31  provides  plots  of  the  MoA  standard  deviation  of  observer  placements  vs.  the  reference 
distance  for  each  of  the  four  test  devices.  The  DP  A  (“RealHD”)  standard  deviation  at  600  cm 
reference  distance  is  about  1 .4  cm.  The  S3D  (“Laptop”)  shows  consistent  performance 
variability  with  a  standard  deviation  of  about  5. 0+0. 2  cm  across  rod  distances.  At  the  600  cm 
reference  distance  the  data  appear  to  suggest  that  two  FoLD  systems  (AIS  and  ZIR  devices) 
support  smaller  (better)  variability  in  placements  at  the  600  cm  target  distance  than;  however, 
variability  climbs  monotonically  as  the  target  is  moved  forward  or  backward  in  depth  from  the 
600  cm  until  they  reach  levels  at  or  just  above  the  S3D. 


Display 


RealHD 


Zebra 


— i - 1 - 1 - 1 - 1 — 

594  597  600  603  606 


Reference  Pin  Distance  (cm) 

Figure  32.  Variability  of  Placement  Performance  vs.  Reference  Distance 
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4.2.1.2.3  Stereoacuity  Threshold  Estimates  Based  on  MoA  Dataset  1 

Stereoacuity  threshold  estimates,  across  participants,  for  each  of  the  four  devices  (viewing 
conditions)  are  tabulated  in  Table  2.  These  thresholds  were  computed  as  binocular  disparity 
values  (in  arcsec)  from  the  means  and  standard  deviations  (S.D.)  of  absolute  errors  in  placement 
performance  assuming  the  average  reference  rod  distance  of  600  cm,  an  average  eye  separation 
of  6.3  cm,  and  crossed  disparity  relative  to  the  reference  rod.  The  threshold  for  the  “angle  of 
stereoscopic  parallax”  (ASP,  aka  stereoacuity  threshold)  is  computed  using  the  binocular 
disparity  equation  given  by  James  (1908)  and  Howard  (1919a)  later  used  by  Cormack  &  Fox 
(1985a):  see  Figure  3a  and  Equation  1.  Thresholds  were  computed  using  both  means  and 
standard  deviations,  as  either  or  both  methods  are  commonly  employed  in  similar  evaluations 
and  can  be  interpreted  as  accuracy  (means)  or  as  precision  (standard  deviations). 

The  computed  thresholds  of  4.74  (means)  and  4.47  (S.D.)  arcsec  or  the  physical  DPA;  these 
values  are  comparable  to  commonly  observed  for  experienced  viewers  with  good  vision  (around 
2-10  arcsec  (Howard,  1919a;  Deyo,  1922;  Fielder  &  Moseley,  1996).  The  rods  were  fixed 
diameter.  However,  the  actual  width  of  the  targets  in  the  two  rod  case  has  been  found  to  have  no 
effect  on  depth  acuity  (Berry,  Riggs,  and  Richards,  1950). 

Values  for  the  three  electronic  displays  were  two  to  three  times  poorer,  suggesting  that  on 
average,  reduced  quality  depth  stimuli  were  being  displayed  and/or  perceived. 

The  computed  MoA  stereoacuity  values  for  the  S3D  (“Laptop”)  device  were  17.08  and  13.49 
arcsec.  It  is  remarkable  that  the  high  image  quality  rendered  by  the  laptop  shown  in  Figure  25 
resulted  in  uniformly  poor  results  compared  to  the  reference  device,  the  physical  DPA.  Other 
work  using  different  S3D  devices  and  test  conditions  have,  however,  indicated  hero  results  as 
low  as  5-10  arcsec.4 

The  computed  stereoacuity  of  the  AIS  (“Perspecta”)  was  9.24  (means)  and  12.56  (S.D.)  arcsec. 
The  computed  stereoacuity  of  the  ZIR  (“Zebra”)  was  12.35  (means)  and  15.34  (S.D.)  arcsec. 
However,  the  visual  artifacts  inherent  these  two  display  systems  shown  in  Figure  26  (AIS)  and 
Figure  27  (ZIR)  may  have  affected  these  results. 


Table  2.  MoA  Stereoacuity  Threshold  Estimates 


Device  Under  Test 

Stereoacuity  Thresholds 
from  Means  (arcsec) 

Stereoacuity  Thresholds 
from  S.D.  (arcsec) 

S3D  (“Laptop”) 

17.08 

13.49 

AIS  (“Perspecta”) 

9.24 

12.56 

DPA  (“RealHD”) 

4.75 

4.47 

ZIR  (“Zebra”) 

12.35 

15.34 

4  *Prior  work  in  our  lab — using  (a)  different  S3D  devices,  (b)  a  different  psychophysical  task  (QUEST  adaptive 
staircase  method  with  2AFC;  Mclntire,  Wright,  Harrington,  Havig,  Watamaniuk,  and  Heft,  2014),  (c)  high-quality 
anti-aliasing  (e.g.,  Lloyd,  2012),  and  (d)  different  stimuli  (three -bar  vs.  two-bar) — produced  stereoacuity  thresholds 
low  as  5-10  arcsec. 
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4.2.2  Dataset  2:  Depth  Acuity  Threshold  Measured  by  Method  of  Constant  Stimuli  (MoCS) 

The  Method  of  Constant  Stimuli  (MoCS)  two-alternative  forced  choice  (2AFC)  judgment  for 
depth  acuity  threshold  measurement  using  two  rods  was  originally  devised  by  James  (1908)  and 
used  by  Howard  (1919a).  Howard,  and  later  Deyo,  introduced  a  “Six-Meter  Stereoscope” 
apparatus  that  was  capable  of  both  the  MoA  as  well  as  the  laborious  data  collection  process 
required  for  large  numbers  of  subjects  using  the  original  MoCS.  Deyo  (1922)  demonstrated  that 
MoA  data  (taken  with  the  Howard-Deyo  DPA  devices  shown  in  Figure  6)  was  as  reliable  as 
MoCS  data  reported  by  Howard  (1919a)  taken  with  Howard  DPA  device  Figures  4. 

The  MoCS  Dataset  2  for  the  present  work  was  collected  for  the  same  test  devices  and  subjects  as 
for  the  MoA  Dataset  1.  The  intent  was  to  further  investigate  the  location-dependent  visual 
artifacts  that  seemed  to  give  observers  an  unintentional  cue  for  matching  depth  locations  on  the 
two  FoLD  systems.  The  goal  was  to  ascertain  if  observers  could  simply  judge  which  of  two 
stimuli  appeared  nearer  in  depth,  on  each  trial  using  pairs  of  stimuli  at  all  pairs  of  the  five  depth 
locations  (594,  597,  600,  603,  and  606  cm)  tested  in  the  MoA  experiment.  In  the  MoA  Dataset  1 
experiment,  viewers  could  have,  and  likely  did,  utilize  the  conspicuity  and  magnitude  of  visual 
artifacts  to  perform  an  artifact-matching  task  for  the  two  FoLD  systems  instead  of  only  a  depth- 
matching  task  as  originally  intended.  Instead  of  using  a  manually-controlled  matching  task 
(MoA),  a  forced  depth  discrimination  task  2FC  was  utilized  so  that  display  artifacts  would  be  of 
little  or  no  help  in  performing  the  judgments,  as  the  artifacts  could  appear  at  either  near  or  far 
locations  with  no  obvious  systematic  pattern.  Presentation  order  of  stimulus  pairs  was 
randomized. 

The  MoCS  Dataset  2,  presented  in  Table  3  and  Figure  33,  shows  that  all  test  devices  were 
capable  of  producing  subject  depth  discrimination  at  a  level  far  above  chance.  However,  the 
correct  rates  for  the  S3D  (72.6%),  AIS  (78.8%),  and  ZIR  (70.7%)  devices  were  all  significantly 
less  than  for  the  physical  DPA  (92.1%).  The  three  electronic  3D  displays  apparently  all  have 
significant  problems  regarding  their  ability  to  present  high-quality  visual  depth  stimuli. 


Table  3.  MoCS  Performance  Averaged  Across  Subjects  for  Four  Test  Devices 


Device  Under  Test 

Percent  Correct  (%)  * 

p -values  ** 

S3D  (“Laptop”) 

72.6 

<.0001 

AIS  (“Perspecta”) 

78.8 

<.0001 

DPA  (“RealHD”) 

92.1 

<.0001 

ZIR  (“Zebra”) 

70.7 

<.0001 

*  Chance  performance  is  50%  using  a  2AFC  method  for  the  MoCS. 
**  Associated  p-values  significant  at  a  <.01  level. 
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Figure  33.  Performance,  Averaged  Across  Viewers,  for  the  S3D  (“Laptop”),  AIS 
(“Perspecta”),  DPA  (“RealHD”)  and  ZIR  (“Zebra”)  Devices  Under  Test;  Error  Bars 

Represent  ±1  SEM 


Next,  we  look  at  how  the  percent  correct  rate  for  each  display  is  modulated  by  different  stimulus 
separations  in  depth,  and  compare  these  magnitudes  to  the  DPA  results.  Figure  34  shows  that,  for 
the  physical  DPA  (“RealHD”),  the  smallest  stimulus  separations  of  only  3  cm  allowed  for  85.0% 
correct  discriminations.  If  the  separations  were  increased  to  a  larger  value  of  6  cm,  giving  a 
larger  depth  cue,  then  performance  essentially  became  perfect  at  99.2%  correct.  The  only  3D 
display  to  even  reach  values  approaching  perfect  performance  was  the  AIS  (“Perspecta”),  which 
required  the  largest  stimulus  separations  of  12  cm  in  order  to  reach  99.2%  correct.  The  stereo 
S3D  (“Laptop”)  performance  maxed  out  at  only  86.7%  with  12  cm  separations,  and  the  ZIR 
(“Zebra”)  maxed  out  even  lower,  at  only  80.0%  correct  for  the  12  cm  separations. 
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Figure  34.  Performance,  Averaged  Across  Viewers,  vs.  Depth  Stimuli  Separations  for  the 
S3D  (“Laptop”),  AIS  (“Perspecta”),  DPA  (“RealHD”)  and  ZIR  (“Zebra”)  Devices  Under 

Test;  Error  Bars  Represent  ±1  SEM 


For  the  6  cm  separations  the  mean  percentage  correct  performance  was  near-perfect  on  the 
physical  DPA  (“RealHD”)  device,  but  just  74.7%  for  the  S3D  (“Laptop”),  80.8%  for  the  AIS 
(“Perspecta”),  and  73.6%  for  the  ZIR  (“Zebra”)  devices. 

For  the  3  cm  separation  (the  most  difficult  discrimination  condition),  the  average  performances 
were  DPA  (85.0%),  S3D  (63.8%),  AIS  (69.4%),  and  ZIR  (63.8%). 

Ideally,  when  using  MoCS,  percent  correct  judgments  must  reach  a  high-enough  magnitude  so 
that  a  75%  correct  threshold  (or  some  selected  above-chance  level  of  performance)  can  be 
computed,  and  compared  across  experimental  manipulations.  In  this  case,  we  wish  to  compare 
stereoacuity  thresholds  across  the  different  display  devices  relative  to  the  DPA.  Unfortunately, 
all  four  devices  cannot  be  directly  compared,  because  some  of  the  displays  have  non-overlapping 
percent  correct  distributions:  the  DPA  smallest  percent  correct  was  85.0%  at  3  cm,  but  the 
highest  performance  for  the  ZIR  device  never  reached  over  80.0%  correct  at  any  separation  level. 

An  indirect  estimate  of  the  threshold  magnitudes  from  MoCS  Dataset  2  can  be  made  as  indicated 
graphically  in  Figure  35,  utilizing  an  80%  correct  threshold  level.  This  indirect  comparison 
gives  the  following  estimates  for  stereoacuity  thresholds  on  each  of  the  devices,  using  the  MoCS 
at  an  80%  correct  level  (Table  4),  and  extrapolating  the  data  for  the  DPA  apparatus: 

The  solid  red  line  indicates  the  80%  correct  performance  threshold.  The  solid  blue  lines  represent 
the  corresponding  thresholds  in  terms  of  stimulus  magnitudes  (with  the  DPA  using  an 
extrapolated  intersection  with  the  80%  performance  line). 
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Figure  35.  Indirect  Estimate  of  Threshold  Magnitudes  from  Data  Plotted  in  Figure  34: 
Intersections  of  80%  Correct  Performance  (Horizontal  Solid  Red  Line)  with  Device 
MoCS  Dataset  2  Performance  (Vertical  Dashed  Blue  Lines);  Error  Bars  ±1  SEM 


Table  4.  Estimated  Stereoacuity  Thresholds  Supported  by  the  Four  Devices  Under  Test 
Computed  Assuming  an  80.0%  Correct  Magnitude  as  Indicated  in  Figure  35 


Device  Under  Test 

Stereoacuity  Threshold 
Estimate  (cm) 

Stereoacuity  Threshold 
Estimate  (arcsec) 

S3D  (“Laptop”) 

9 

33.0 

AIS  (“Perspecta”) 

6 

21.9 

DPA  (“RealHD”) 

~1.5 

5.4 

ZIR  (“Zebra”) 

12 

44.2 

The  data  suggest  that  stereoacuity  thresholds  were  best  on  the  DPA  apparatus,  of  a  magnitude  we 
would  expect  at  5.4  arcsec,  indicating  high-quality  depth  perception.  Thresholds  were 
considerably  worse  on  all  three  electronic  3D  displays,  ranging  from  22  to  44  arcsec.  These  data 
suggest  that  all  three  of  the  electronic  3D  displays  supported  depth  perception  at  levels  four  to 
nine  times  worse  than  the  DPA. 

The  results  of  this  MoCS  Dataset  2  experiment  were  generally  in  accord  with  the  previous  MoA 
Dataset  1  experiment.  Interestingly,  despite  the  change  in  psychophysical  methodology,  the 
present  data  still  appear  to  suggest  that  the  two  FoLD  systems  were  capable  of  producing  depth 
stimuli  of  a  quality  comparable  to  the  stereo  laptop  despite  their  apparently  poorer  image  quality 
and  presence  of  visual  artifacts. 
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4.2.3  Dataset  3:  Depth  Acuity  Threshold  Measurement  by  MoCS  for  Monocular  Observers. 

The  MoCS  2AFC  Dataset  3  was  collected  for  five  observers  using  only  one  eye  to  perform  the 
task.  The  dominant  eye  was  utilized;  the  non-dominant  eye  was  occluded  by  an  eye -patch.  The 
purpose  of  this  follow-up  data  collection  was  to  unequivocally  confirm  or  deny  whether  the  two 
FoLD  systems  were  providing  non-stereopsis  cues  to  depth  (i.e.  location-dependent  visual 
artifacts),  and  to  compare  the  FoLD  systems  with  the  S3D  for  which  high-quality  depth  cues  had 
been  expected.  A  MoCS  2AFC  stereoscopic  vision  task  should  theoretically  result  in 
performance  only  at  the  level  of  chance  (50%)  for  observers  using  only  one  eye.  Any  deviations 
from  chance  levels  would  indicate  the  presence  of  unintentional  but  systematic  monocular  cues 
to  depth/location.  Similar  techniques  have  been  used  by  Bohr  &  Read  (2013)  and  Holmes  & 
Fawcett  (2005)  to  control  for  possible  monocular  (non-stereo)  cues  present  in  standard 
optometric  tests  of  stereo  acuity,  with  the  assumption  that  if  binocular  stereoacuity  is  identical  to 
the  “monocular  stereoacuity”  measure,  then  the  binocular  vision  status  should  be  clinically  rated 
as  “stereo-negative”  or  stereo-blind. 

An  early  study  comparing  monocular  to  binocular  performance  on  the  Howard-Deyo  DPA 
illustrated  in  Figure  6  showed  that  binocular  performance  was  over  six  times  more  accurate 
(using  a  MoA  technique)  than  monocular  performance  (Deyo,  1922). 

The  data  in  Table  5  show,  somewhat  surprisingly,  that  performance  on  all  four  viewing  systems 
was  different  than  chance  would  predict,  when  using  a  0.05  significance  level.  These  results 
appear  to  suggest  that  non-binocular  (monocular)  cues  to  depth  exist  in  all  systems.  This  result 
was  thought  to  be  (a)  not  possible  in  the  case  of  the  S3D  (“Laptop”)  as  all  possible  monocular 
cues  were  thought  to  be  strictly  controlled,  and  (b)  unlikely  in  the  case  of  the  DPA  (“RealHD”) 
device  as  the  theoretical  possibility  of  horizontal  size  cues  available  from  the  width  of  the  rods  as 
a  monocular  depth  source  has  been  studied  (e.g.  Howard  1919a,  Woodbume,  1934).  The  use  of  a 
more-conservative  alpha  value  of  0.01  produces  results  that  suggest  that  only  the  two  FoLD 
systems  were  providing  for  greater-than-chance  performance.  Thus,  the  0.01  significance  level 
implies  the  FoLD  systems  (AIS,  ZIR,  but  not  the  S3D  nor  the  DPA  apparatus,  were  providing 
non-stereoscopic  but  depth-related  information  to  viewers  which  improved  their  performance 
above  chance,  perhaps  because  of  the  visual  artifact  cues,  which  are  visible  monocularly  and 
whose  presence  and  magnitude  scales  in  depth. 


Table  5.  Performance  across  Five  Subjects  when  Devices  Were  Viewed  Monocularly.* 


Device  Under  Test 

Monocular 
Percent  Correct 

(%) 

Sample  size 
(total  trials) 

p -values 

S3D  (“Laptop”) 

48.8 

1000 

.031** 

AIS  (“Perspecta”) 

55.0 

1000 

.003*** 

DPA  (“RealHD”) 

55.5 

200 

.017** 

ZIR  (“Zebra”) 

54.4 

1000 

.005*** 

*  Chance  performance  is  50%  using  a  2AFC  method  for  the  MoCS. 
**  Associated  p- values  significant  at  a  <.05  level. 

***  Associated  p- values  significant  at  a  <.01  level. 
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While  monocular  performance  on  the  AIS  (“Perspecta”)  and  ZIR  (“Zebra”)  were  significantly 
above  chance  at  55.0%  and  54.4%,  respectively,  these  were  not  alarmingly  high  percent  correct 
values.  When  the  same  task  was  performed  binocularly,  percent  correct  values  tended  to  be  a  bit 
higher  using  both  systems  (78.8%  and  70.7%,  respectively).  This  suggests  that  these  systems 
were  at  least  providing  some  stereoscopic  depth  cues  which  helped  performance  over  the 
monocular  condition,  but  only  of  a  smaller  relative  magnitude  than  some  of  the  previous 
analyses  (Deyo,  1922)  had  documented.  The  binocular-to-monocular  advantage  for  the  S3D 
(“Laptop”)  and  the  volumetric  AIS  (“Perspecta”)  were  the  same  on  average,  giving  +23.8% 
improvements  in  percent  correct  judgments,  while  the  binocular  advantage  for  the  ZIR  (“Zebra”) 
was  only  a  +16.3%  improvement.  In  contrast,  the  binocular  improvement  for  the  physical  DPA 
(“RealHD”)  was  considerably  larger  at  +36.6%,  suggesting  that  all  three  3D  devices  were 
incapable  of  producing  high  quality  binocular  cues  to  depth  using  a  virtual  James  task,  when 
compared  to  viewing  the  real  apparatus. 


Table  6.  Performance  across  Subjects:  Binocular  vs.  Monocular 


Device  Under  Test 

Binocular 
Percent  Correct 

(%) 

Monocular 
Percent  Correct 

(%) 

Binocular 
Advantage  (% 
difference) 

S3D  (“Laptop”) 

72.6 

48.8 

+23.8 

AIS  (“Perspecta”) 

78.8 

55.0 

+23.8 

DPA  (“RealHD”) 

92.1 

55.5 

+36.6 

ZIR  (“Zebra”) 

70.7 

54.4 

+16.3 

An  examination  of  the  monocular  percent  correct  performance  as  a  function  of  reference  pin 
position,  a  different  story  emerges.  Figure  36  shows  the  performance  data  plotted  in  this  manner. 
The  DPA,  with  only  one  reference  pin  position  at  600  cm,  shows  monocular  performance  at 
about  chance  level  (the  solid  red  line),  which  is  a  straightforward  finding.  The  S3D  (“Laptop”) 
supported  performance  roughly  at  chance  across  all  reference  rod  locations,  with  a  flat 
performance  line  as  would  be  expected.  However,  percent  correct  performance  for  the  two 
FoLD  systems  was  approximately  linear,  but  far  from  flat,  and  systematic  biases  in  responses  are 
obvious.  Both  FoLD  systems  resulted  in  a  strange  performance  pattern:  when  a  reference  rod 
was  located  at  the  nearest  positions,  participants  almost  always  guessed  incorrectly;  when  a 
reference  rod  was  located  at  the  central  position,  performance  was  at  chance;  and  when  a 
reference  rod  was  located  at  the  farther  positions,  performance  climbed  dramatically  and  became 
almost  perfect.  This  provides  further  confirmation,  importantly,  that  systematic  monocular  cues 
to  depth  locations  were  present  in  the  FoLD  systems,  which  biased  performance  (above  or  below 
chance)  whenever  the  reference  pin  was  located  off  the  central  depth  position  of  the  displays. 
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Figure  36.  Performance  vs.  Reference  Pin  Location;  Error  Bars  Represent  ±1  SEM. 


This  pattern  would  manifest  if  observers  had  adopted  a  decision  strategy  like:  “If  the  reference 
pin  is  blunder  and/or  dimmer  than  the  other  pin,  it  must  be  farther  away.”  This  would  result  in 
very  high  accuracy  when  the  reference  pin  was  farther  away,  and  very  poor  accuracy  when  the 
pin  was  near  (because  the  blur  and  dimming  effects  occur  at  both  near  and  far  locations  relative 
to  the  center  position).  Thus,  this  almost  “perfectly- wrong”  combined  with  almost  “perfectly 
right”  performance  pattern  likely  resulted  from  mis-perceptions  in  the  depth  locations  due  to  the 
location-dependent,  monocular  visual  artifacts  in  these  systems,  as  described  earlier.  These 
intriguing  results  explain  why  performance  on  the  FoLD  systems  could  be  both  noticeably  better 
(and  simultaneously  worse)  than  chance  would  predict  when  performing  a  binocular  depth 
discrimination  task  with  only  one  eye. 

If  one  next  examines  the  binocular  advantage  of  each  display  system  across  comparison  pin 
locations  (with  reference  pin  always  fixed  at  600  cm,  n=200  per  display),  the  results  again 
confirm  that  the  two  FoLD  systems  were  causing  systematic  near/far  confusions  (see  Table  7; 
data  shown  graphically  in  Figure  37).  The  confusions  seem  to  be  resulting  in  extremely  large 
binocular  advantages  for  near  comparison  pin  locations,  which  are  just  too  good  to  be  true 
(beating  out  the  DPA  for  some  distances).  The  converse  was  true  for  far  pin  locations,  for  both 
FoLD  systems,  resulting  in  huge  binocular  disadvantage s\ 

Table  7  shows  performance  across  subjects  and  for  each  comparison  pin  distance  (reference  pin 
fixed  at  600  cm).  Sample  size  for  each  display  is  200  trials;  for  each  distance  is  50  trials. 

Average  binocular  performance  is  compared  to  monocular  performance,  and  the  binocular-to- 
monocular  advantages  are  provided. 
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Table  7.  Average  Binocular  vs.  Monocular  Performance 


Device  Under 
Test 

Comparison 
Pin  Distance 
(cm) 

Monocular 

Percent 

Correct 

Binocular 

Percent 

Correct 

Binocular 

Advantage 

(%) 

Average 

Binocular 

Advantage 

S3D 

(“Laptop”) 

594 

40 

68 

+28 

+21.3% 

597 

52 

52 

0 

603 

52 

70 

+18 

606 

44 

83 

+39 

AIS 

(“Perspecta”) 

594 

2 

68 

+66 

+18.5% 

597 

14 

58 

+44 

603 

92 

70 

-22 

606 

96 

82 

-14 

DPA 

(“RealHD”) 

594 

58 

100 

+42 

+36.5% 

597 

62 

83 

+21 

603 

56 

87 

+31 

606 

46 

98 

+52 

ZIR 

(“Zebra) 

594 

5 

72 

+67 

+16.0% 

597 

10 

73 

+63 

603 

86 

53 

-33 

606 

96 

63 

-33 

Collapsing  the  average  binocular  advantages  across  pin  locations  gives  a  more  general 
comparison  of  the  displays  (rightmost  column  in  Table  7),  with  the  data  confirming  initial, 
intuitive  suspicions:  the  DPA  gave  the  largest  binocular  advantage  of  +36.5%,  followed  by  the 
S3D  (“Laptop”)  at  +21.3%.  Poorer  still  were  the  two  FoLD  systems,  with  the  AIS  (“Perspecta”) 
volumetric  device  at  +18.5%  just  besting  the  ZIR  (“Zebra”)  holographic  display  at  +16.0%. 

In  Figure  37  the  solid  performance  line  indicates  binocular  performance  on  the  2AFC  task;  the 
dashed  line  indicates  corresponding  monocular  performance  on  the  same  task.  Regions  of 
positive  binocular  advantage  are  shaded  green;  regions  of  binocular  disadvantage  (negative)  are 
shaded  red.  Notice  that,  for  monocular  viewing,  the  AIS  (“Perspecta”)  and  ZIR  (“Zebra”) 
devices  were  biasing  performance  across  pin  positions,  sometimes  helping  but  sometimes  hurting 
performance,  which  is  likely  due  to  the  unique  FoLD  system  artifacts  such  as  blur  and/or 
dimming  being  interpreted  as  depth  cues  for  far  distance. 

This  analysis  confirms:  (a)  that  the  physical  DPA  (“RealHD”)  provides  the  highest  quality 
binocular  depth  cues  for  determining  pin  positions;  (b)  that  the  S3D  (“Laptop”)  was  providing 
high  quality  binocular  disparity  cues  to  depth,  which  were  not  apparent  in  previous  analyses;  and 
(c)  that  the  two  FoLD  systems  provided  for  some  binocular  cues,  with  positive  overall  binocular- 
over-monocular  advantages,  but  the  quality  of  such  cues  appears  to  have  been  degraded  relative 
to  the  S3D  (“Laptop”)  and  severely  degraded  relative  to  the  physical  DPA  (“RealHD”). 
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Figure  37.  Performance,  Averaged  Across  Viewers,  vs.  Comparison  Bar  Location  for 
Reference  Bar  Fixed  at  600  cm;  Error  Bars  Represent  ±1  SEM 


Howard  tried  10  men  and  found  that  the  depth  difference  threshold  varied  from  an  average  of  13 
mm  with  both  eyes  open  to  over  285  mm  when  only  one  eye  was  open  (Howard,  1919  TAOS,  p. 
234). 


65 

DISTRIBUTION  STATEMENT  A.  Approved  for  public  release:  distribution  unlimited. 


88ABW  Cleared  09/08/2016;  88ABW-20 16-4404. 


4.3  Effects  of  Rod  Lighting  Model  Used  to  Render  Rod  Images  for  View  on  Devices 

4.3.1  Unexpectedly  High  Stereoacuity  Thresholds  Observed  for  S3D  (“Laptop”)  Device 

Throughout  the  human  subject  data  collection  and  analysis  the  comparatively  poor  stereoacuity 
thresholds  supported  by  the  S3D  (“Laptop”)  device  vs.  those  observed  with  the  FoLD  systems, 
was  difficult  to  understand.  The  mean  stereoacuity  threshold  on  the  S3D  (“Laptop”)  device  was 
measured  at  17.1  arcsec  by  MoA  and  33.0  arcsec  by  MoCS.  These  are  not  terrible  stereoacuity 
values,  but  they  are  not  excellent,  and  this  was  confusing/frustrating  for  several  reasons. 

First,  stereoacuity  thresholds  measured  for  other  S3D  displays  (in  this  and  other  labs)  have 
consistently  been  lower  (as  low  5-10  arcsec),  which  is  comparable  to  the  lowest  (best) 
stereoacuity  thresholds  (for  physical  DPA  devices)  of  2  or  3  arcsec  (Howard,  1919a)  (Fielder  & 
Moseley,  1996).  The  mean  stereoacuity  threshold  obtained  with  the  physical  Howard-DPA 
device  (“RealHD”)  used  in  the  present  study  was  4.8  arcsec  by  MoA  and  5.4  arcsec  by  MoCS. 

Second,  the  observed  stereoacuities  for  the  S3D  (“Laptop”)  device  were  similar  in  magnitude  to 
those  for  the  two  FoLD  systems,  which  ranged  from  9.2  arcsec  by  MoA  for  the  AIS 
(“Perspecta”)  device  to  44.2  arcsec  by  MoCS  for  the  ZIR  (“Zebra”)  device. 

Third,  the  S3D  (“Laptop”)  had  image  quality  superior  to  that  of  either  FoLD  system.  The 
observation  of  comparable  stereoacuity  thresholds  was,  thus,  surprising  and  puzzling.  Part  of  the 
explanation  probably  lies  in  the  FoLD  systems  providing  for  monocular  depth  information  due  to 
their  location-dependent  visual  artifacts.  Attempts  to  control  for  differential  image  quality  and 
for  monocular  clues  were  undertaken  in  the  collection  and  analysis  of  MoCS  Dataset  2  and 
MoCS  Dataset  3;  however,  these  attempts  were  only  partially  successful.  The  MoCS  Dataset  3 
study  showed  that,  assuming  a  conservative  alpha  value,  monocular  depth  cues  impacted  results 
obtained  for  FoLD  systems,  but  not  for  the  S3D  (“Laptop”)  or  physical  DPA  (“RealHD”).  In 
addition,  the  third  study  showed  that  when  the  reference  pin  was  held  at  a  constant  position,  the 
magnitude  of  the  binocular-to-monocular  advantage  was  +21.3%  for  the  S3D  (“Laptop”)  device 
compared  to  +18.5%  for  the  AIS  (“Perspecta”)  and  +16.0%  for  the  ZIR  (“Zebra”).  Yet  another 
notable  advantage  of  the  S3D  (“Laptop”),  demonstrated  in  the  analyses  of  all  three  datasets,  was 
that  it  did  not  suffer  the  problematic  near-to-far  perceptual  confusions  elicited  by  the  FoLD 
systems. 

Although  the  presence  of  visual  artifacts  in  the  FoLD  systems  might  help  explain  their 
surprisingly  decent  stereoacuity  values,  it  does  not  seem  to  explain  the  relatively  poor  values 
obtained  on  the  S3D  (“Laptop”)  device.  This  result  may  have  been  due  to  the  nature  of  the 
visual  stimuli  used  in  our  studies.  Specifically,  in  the  conceptual  lighting  model  of  the  virtual 
rods,  the  virtual  light  source  was  coming  from  a  point  source  directly  in  front  of  the  rods  (i.e 
from  the  direction  of  the  viewer). 

The  use  of  front-lit  white  rods  lit  from  a  single  point  source  displayed  against  an  empty  black 
background  in  the  present  work  resulted  in  rendered  images  of  rods  “reflecting”  most  intensely 
from  their  centers  towards  the  viewer,  with  a  prominent  luminance  degradation  moving  laterally 
off  the  center  of  the  vertical  rods,  going  gradually  from  bright  white,  to  grey,  to  greyer,  to  black 
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as  shown  for  the  S3D  (“Laptop”)  stimulus  pictured  in  Figure  38.  The  pixel  structure  of  the 
display  is  visible.  As  can  be  seen,  the  fading  from  brightest  to  darkest  at  the  left  and  right  edges 
of  the  stimuli  spans  several  pixels  in  width,  potentially  creating  sub-optimal  stimuli  for 
stereoacuity  threshold  measurements.  Additionally,  thresholds  are  generally  best  when  tested 
with  image  content  possessing  high  spatial  frequencies  and  sharp  contours.  However,  in  the 
present  work  the  automatic  anti-aliasing  employed  by  the  NVidia  software  was  turned  on  and, 
although  the  exact  specifications  of  the  anti-aliasing  function  are  unclear,  may  have  “blurred” 
any  sharp  edges  even  further. 


Figure  38.  Close-up  Photographic  of  Front-Lit  Rod  Stimulus  Displayed  on  S3D  (“Laptop”) 


These  issues  may  have  resulted  in  degradations  of  the  crisp,  sharp,  high-contrast  edges  that  the 
visual  system  needs  to  perform  extremely  precise  stereoscopic  localization.  For  instance, 
Patterson  (2009,  p.  992)  notes  that  “fine  sensitivity  to  small  disparity  differences  require  a  high 
spatial  frequency  (fine  details  in  an  image).” 
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4.3.2  Follow-Up  Data  Collection  with  Lighting  Model  Changes 


To  test  whether  the  front  lighting  model  might  have  been  inadvertently  “blurring”  the  stimulus 
edges  too  much  in  the  S3D  (“Laptop”)  device  (reducing  the  sharp,  high  spatial  frequency 
countours),  an  abbreviated  follow-up  data  collection  was  accomplished  with  one  experienced 
subject  with  excellent  stereovision.  In  this  follow-up  collection  the  rocl/background  polarity  was 
changed  so  that  there  were  now  black  rods  with  crisp  edges  --  and  with  no  grey  gradations  -- 
against  a  solid  bright  white  background.  Some  40  trials  were  collected  in  this  reversed  polarity 
condition  by  MoA  and  compared  to  the  average  absolute  value  threshold  for  the  same  observer 
in  the  60  trials  from  the  MoA  Dataset  1  experiment. 

The  stereoacuity  threshold  for  this  single  observer  improved  dramatically,  dropping  nearly  in 
half  from  3.83  cm  (13.7  arcsec)  to  2.26  cm  (8.1  arcsec),  which  is  in  line  with  previous  experience 
with  S3D  devices. 

These  complicating  issues  are  described  along  with  a  description  of  how  they  were  dealt  with  in 
the  present  work  so  that  lessons  learned  can  be  taken  into  account  to  control  or  avoid  such 
problems  in  future  work. 

4.3.3  Lighting  Approaches  in  Seminal  Stereoacuity  Research  with  Physical  DPA  Devices 

It  is  noted  that  the  earliest  stereoacuity  measurements  with  physical  DPA  devices  using  two 
vertical  rods  involved  two  white  rods  lit  uniformly  from  cdl  subject-viewable  directions  against  a 
black  velvet-covered  background  (James  1907,  James  1908).  The  present  electronically 
stimulated  white  rods  were  lit  from  a  single  modeled  point  source  for  the  collection  of  Datasets 
1,  2  and  3,  which  resulted  in  the  non-uniform  appearance  shown  in  Figure  38  and  in  the  ill- 
defined  edges.  If  front-lit  models  of  white  rods  are  used  in  future  depth  perception  research  on 
S3D  or  FoLD  systems  (Datasets  1,  2,  3  in  the  present  work  summarized  in  subsection  4.3.1),  care 
should  be  taken  that  the  lighting  model  involve  uniform  illumination  of  the  subject-side  of  the 
rods  rather  than  a  point  source. 

Howard  (1919a)  reversed  the  colors  in  his  version  of  the  James  apparatus:  two  black  rods  back¬ 
lit  uniformly  and  indirectly  backlit  by  light  reflected  off  a  white  cardboard  background.  The  back 
was  the  only  white  interior  surface;  all  other  interior  surfaces  were  painted  dead  black.  Howard 
found  14  of  106  observers  examined  possessed  an  average  stereoacuity  threshold  of  1.9  ±0.1 
arcsec  (Howard,  1919a  TAOS,  p.  214).  Howard  further  concluded  that  normal  judgement  of 
distance  was  <8.0  arcsec  (the  stereoacuity  depth  equivalent  of  20/20  Snellen  lateral  acuity).  The 
present  follow-up  data  collection  (summarized  in  subsection  4.3.2)  is  an  electronic  version  of  the 
Howard  lighting  approach. 
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4.4  Computer-Rendered  Images  Issues  for  the  Electronic  James  Depth  Acuity  Test 

The  original  concept  of  this  effort  was  to  recreate  the  James  depth  acuity  test  in  electronic  form. 
However,  each  electronic  display  device  used  in  the  present  work  encountered  unique  difficulties 
in  replicating  the  physical  DPA  presentation  of  the  two-bar  stimuli.  The  inability  to  find  a  single 
virtual  model  that  could  be  used  across  all  electronic  displays  (three  pictured  in  Figure  39  along 
with  the  AFRL-modified  Howard  DPA  (Howard  1919b)  was  initially  frustrating.  The  James 
rods  used  in  the  Howard  and  Howard-Deyo  DPAs  (Howard  1919a,  1919b;  Deyo  1922)  are  an 
extremely  simple  research  stimulus.  An  inability  to  generate  easily  comparable  virtual  rods 
across  electronic  displays  suggests  that  cross-category  display  experiments  using  more  subtle, 
elaborate  or  generalizable  stimuli  will  be  challenging. 

Optimal  rod  appearance  was  dependent  on  the  unique  characteristics  of  each  display. 

The  Howard  DPA  (Howard,  1919a,  1919b)  used  black  rods  uniformly  back-lit  (no  side  of  front 
lighting  whatsoever).  However,  the  black  rods  in  the  AFRL-modified  Howard  DPA  (Howard, 
1919b)  were  sandwiched  between  two  long  fluorescent  tubes  so  that  the  sides  of  the  rods 
received  greater  illumination  than  the  front,  creating  a  subtle,  darker  in  the  middle,  brightness 
change  across  the  width  of  the  rod.  Illumination  also  varied  with  the  position  of  the  moveable 
rod,  but  the  variation  was  deemed  too  small  to  serve  as  a  cue  to  rod  position.  The  viewing 
window  was  effective  at  reducing  all  other  monocular  depth  cues  with  the  possible  exception  of 
rod  width  (which  has  been  discounted  in  prior  works)  and  small  motion  parallax  cues  (which 
Howard  had  eliminated  by  use  of  a  chin-mount).  Monocular  performance  on  the  AFRL- 
modified  Howard  DPA  (Howard,  1919b)  was  shown  to  be  at  chance  level  on  a  2 AFC  near/far 
depth  judgment,  confirming  the  absence  of  monocular  cues. 

Automation  of  the  Howard  DPA  system  enabled  commonality  with  the  electronic  displays  on 
several  experimental  design  variables  to  include:  subject  input  device,  trial  and  intra-trial 
duration,  and  psychophysical  measurement  method.  The  lack  of  flexibility  in  lighting 
conditions,  rod  reflectance  and  illumination  non-uniformities,  and  background  reflectance  made 
it  difficult  to  compare  the  performance  of  the  physical  AFRL-modified  Howard  DPA  (Howard, 
1919b)  against  electronic  displays  that  could  not  replicate  the  physical  fixed  luminance 
relationships  used  in  the  seminal  works  (James,  1908)  (Howard,  1919a,  1919b)  (Deyo,  1922). 

The  S3D  (“Laptop”)  device  had  good  luminance,  contrast,  and  resolution  and  could  satisfactorily 
produce  a  wide  variety  of  rod  presentations  including:  black  rods  against  a  white  background, 
white  rods  against  a  black  background  and  every  rod/background  contrast  in-between.  In 
addition,  the  large  gray  scale  supported  a  wide  variety  of  virtual  lighting  presentations,  enabling 
subtle  highlighting  and  great  flexibility  in  optimizing  the  surface  appearance  of  the  rods. 
Replicating  the  box  and  window  of  either  the  James  DPA  or  the  Howard  DPA  was  more 
difficult. 

The  physical  AFRL-modified  Howard  DPA  has  a  flat,  uniform  black  appearance  to  the  front  face 
of  the  box  with  a  uniform  white  interior.  Unfortunately,  in  stereo  displays,  uniform  surface 
appearance  produces  ambiguous  binocular  disparity  cues;  without  the  addition  of  significant 
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surface  texture,  the  appearance  of  the  virtual  box  was  unstable  in  depth.  Thus,  all  S3D  displays 
rely  on  a  false  depth  cue  (texture)  to  evoke  a  perception  of  depth.  However,  ultimately  it  was 
concluded  that  all  of  the  electronic  displays  had  difficulty  replicating  the  surrounding  box  face 
and  it  was  eventually  dropped  from  all  electronic  stimulus  presentations,  leaving  only  the  virtual 
rods. 

One  of  the  critical  functions  of  the  window  on  the  front  of  the  physical  DPA  is  to  eliminate 
vertical  size  as  a  depth  cue.  In  the  S3D  device,  the  vertical  size  of  the  rods  should  not  be  a 
problem,  as  size  is  easily  held  constant  in  rod  image  rendering.  Nevertheless,  when  a  subject 
views  the  rods  stereoscopically,  changing  the  disparity  of  the  rod  image  changes  the  both  the 
perceived  depth  location  and  the  perceived  size  (i.e.,  size-disparity  perceptual  scaling  was  a 
potential  issue).  The  variable  size  perception  is  substantial  and  researchers  may  feel  inclined  to 
correct  the  perception  by  introducing  a  vertical  scaling  factor  to  counter  the  perceived  change  in 
size  induced  by  changing  disparity.  This  inclination  should  generally  be  ignored  for  S3D 
devices  as  it  introduces  a  large  monocular  cue  to  depth.  These  effects  are  a  potential  issue  for 
computer  rendered  stimuli  for  FoLD  devices,  but  they  were  long  ago  ruled  out  in  as  important 
factors  in  the  use  of  well-designed  physical  DPA  devices. 

Discrete  sampling  of  continuous  vector  models  creates  challenges  for  using  S3D  devices  in 
psychophysical  measurements.  At  current  display  resolutions  and  typical  viewing  distances  it  is 
arguable  whether  a  stereo  display  can  be  considered  to  produce  a  continuous  image.  It  is 
commonly,  but  mistakenly,  assumed  that  a  display  with  a  pixel  pitch  of  less  than  1  arcmin  (60 
arcsec)  is  “eye-limited”  and,  therefore,  produces  a  perceptually  continuous  image.  In  reality,  the 
hyperacuity  threshold  is  5  arcsec.  Stereoacuity  measured  on  physical  DPA  devices  has  been 
routinely  measured  at  2  to  6  arcsec.  However,  producing  a  2  to  6  arcsec  pixel  pitch  display  for 
psychophysical  measurements  is  not  possible  and  typically  is  not  necessary.  Anti-aliasing  allows 
a  manipulation  of  relative  pixel  luminance  between  stereo  pixel  pairs.  This  luminance  shift  has 
been  demonstrated  to  support  sub-pixel  stereoacuity  measurements  (Lloyd,  2012). 

Unfortunately,  enabling  anti-aliasing  introduces  ambiguity  into  the  description  of  the  stimulus. 

In  many  instances  the  anti-aliasing  algorithm  used  by  a  particular  S3D  device  is  proprietary  and 
unknown.  It  is  also  unreasonable  to  expect  display  calibration  to  be  routinely  conducted  at  the 
single  pixel  level.  But  in  psychophysical  measurements  a  rigorous  description  of  the  stimulus  is 
considered  essential  to  the  experimental  process.  Until  a  calibration  process  is  established  for 
anti-aliasing,  researchers  will  be  left  with  the  choice  of  accepting  some  ambiguity  in  describing 
or  controlling  the  stimulus,  or  left  with  limiting  the  stimulus  presentation  to  undesirably  coarse 
single-pixel  steps.  These  coarse  steps,  if  not  identified  and  managed  appropriately,  can  cause 
difficulty  in  interpreting  results.  For  example,  in  the  MoCS  psychophysical  method,  if  a 
researcher  picked  two  disparity  levels  from  the  virtual  model  that  were  unknowingly  binned  to 
the  same  pixel  disparity  by  the  computer  driving  the  S3D  device,  the  resulting  cumulative 
distribution  function  would  inappropriately  appear  to  plateau  or  possibly  reverse.  Picking  a 
constant- stimuli  interval  larger  than  the  largest  pixel  step  of  an  S3D  device  should  avoid  this 
problem. 
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Figure  39.  Close-up  of  Sample  Images  (Stimuli  Viewed  by  Subjects  at  6m)  on  (A)  Physical  Depth  Perception  Apparatus 
(DPA),  (B)  Active-Eyewear  Stereo  3D  -  Samsung  22-in.  Notebook  Display  with  NVIDIA  Active-Eyewear  (aeS3D),  (C) 
Actuality  Rotating- Screen  Integral- Slice  Actuality  Integral  Slice  (AIS)  Perspecta  Volumetric  Display  (ARS),  and  (D)  Zebra 

Integral-Ray  Hogel-Based  Zscape  Motion  Display  (ZIR) 
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After  the  James  Test  was  implemented  and  calibrated  on  the  S3D  device,  pre-experimental  trials 
were  run  to  explore  the  range  of  half-image  pixel  shifts  that  could  be  reasonably  investigated 
using  the  procedure.  The  empty  background  field  allowed  for  binocular  fusion  of  very  large 
pixel  shifts,  but  at  large  rod  half  image  shifts,  it  was  noticed  that  the  perceived  depths  no  longer 
matched  the  predictions  from  the  binocular  disparity  equations.  One  potential  explanation  for 
the  poor  correlation  between  predicted  and  perceived  depth  is  the  weakness  of  vergence  eye 
movements  in  producing  depth  cues,  compounded  by  incongruous  head  motion  parallax,  size  and 
focus  cues  unchanging  with  vergence  changes,  etc.  Small  half  image  shifts  can  be  easily  seen 
and  processed  into  accurate  relative  depth  cues  by  the  visual  system  but  large  shifts  require 
vergence  eye  movements  to  maintain  binocularity.  The  binocular  disparity  equations  assume  that 
the  vergence  distance  is  known,  but  vergence  eye  movements  are  weak  depth  cues  and  are 
ineffective  in  determining  distance  without  reinforcement  by  monocular  depth  cues  (Howard,  I., 
2008)  (Mon-Williams  et  al.,  2000).  Of  course  the  James  DPA  (and  all  incarnations  since)  was 
intentionally  designed  to  minimize  monocular  depth  cues  (James,  1908).  The  inadequacy  of  eye 
vergence  to  support  accurate  depth  perception  is  also  aggravated  by  the  inability  of  S3D  devices 
to  produce  appropriate  head  motion  parallax  cues,  a  strong  depth  cue  (Ichikawa  and  Saida, 

1996).  The  larger  the  pixel  shift  presented  on  the  display,  the  worse  the  conflict  between 
disparity  and  head  motion  parallax  cues. 

S3D  displays  can  accurately  produce  relative  object  motion  parallax  (just  not  viewer- to- object 
parallax  without  head/eye  tracking).  Therefore,  it  is  essential  to  maintain  the  correct  virtual 
camera/model  and  subject/display  viewing  geometries  or  else  the  horizontal  separation  between 
the  two  rods  will  vary  with  depth,  creating  a  strong  monocular  depth  cue.  This  viewing 
geometry  sensitivity  is  largely  ignored  in  the  entertainment  industry  and  may  not  be  appreciated 
by  researchers  using  electronic  displays  as  an  inexpensive  tool  to  generat  experimental  stimuli. 

The  AIS  swept  volume  display  could  not  produce  occlusive  surfaces  and  consequently  it  was 
unable  to  display  a  dark  rod  against  a  white  background.  For  the  same  reason,  it  was  unable  to 
reproduce  the  occlusive  features  of  the  physical  DPA  box  and  window.  However,  a  physical 
window  roughly  analogous  to  the  windows  used  in  the  DPA  might  be  utilized  if  desired.  The 
loss  of  the  window  results  in  an  inability  to  control  for  vertical  size  cues.  In  a  swept  volume 
display  the  rod  image  physically  moves  in  depth  and  therefore  the  monocular  angular  subtense  of 
the  rod  in  the  visual  field  changes.  Other  monocular  cues,  such  as  head  motion  and  object 
motion  parallax  depth  cues  are  present  in  the  AIS  (“Perspecta”)  device  and  need  to  be  considered 
in  the  experimental  design.  The  inability  to  produce  occlusive  surfaces  creates  novel  options  for 
rod  appearance.  In  addition  to  the  normal  lighting  options  provided  in  the  virtual  model,  the  rod 
can  be  presented  with  a  white  surface  and  a  hollow  (dark)  center  or  a  filled  in  (bright)  center. 
When  viewed  perpendicular  to  the  rods  axis,  the  luminance  of  the  hollow  rod  appeared  fairly 
uniform  whereas  the  filled  rod  appeared  was  slightly  brighter  in  the  center  and  dropping  off 
gradually  towards  the  edge.  Even  after  calibration,  the  AIS  swept  volume  display  has  some 
residual  non-  uniformity  in  the  viewing  volume,  and  had  an  uncorrectable  difference  in 
brightness  as  the  function  of  radial  distance  from  the  axis  of  rotation.  Thus,  change  in  luminance 
became  a  strong,  but  confusable  depth  cue.  In  the  AIS,  curvature  and  spinning  screen  motions, 
along  with  the  variable  reflections  in  the  two  concentric  domes  around  the  screen,  created 
artifacts  that  also  caused  variable  contrast  as  bars  were  moved  fore  or  aft  of  the  spin  axis. 
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The  ZMD  (“Zebra”)  device  is  a  very  early  proof-of-concept  demonstration  of  a  light-field 
display  and  as  such  it  was  assumed  that  implementing  the  James  test  on  it  would  be  difficult. 

For  example  the  ZMD  image  quality  varies  with  depth:  the  further  the  stimulus  is  presented  from 
the  display  surface  (as  defined  by  the  pixel  layer)  the  blunder  images  get.  Consequently,  blur 
became  a  strong  cue  to  rod  location  in  depth.  This  was  mitigated  to  some  extent  in  the  present 
work  by  using  a  rod  lighting  model  that  resulted  in  a  bright  center  but  dim  edge  appearance  to 
the  rod.  In  effect,  the  lighting  model  mimicked  the  line  spread  function  appearance  for  a  poorly 
focused  line. 

The  blur  also  impeded  the  ability  to  recreate  the  visual  appearance  of  the  physical  DPA  box. 
Light-field  displays  are  similar  to  swept- volume  displays  in  that  a  change  in  object  distance 
results  in  a  change  in  angular  size  of  the  retinal  image,  which  is  a  very  strong  monocular  depth 
cue.  Unlike  swept- volume,  light-field  displays  support  occlusion  and  theoretically  the  box  and 
window  approach  to  dealing  with  vertical  size  variations  should  apply.  Unfortunately  in  the 
ZMD  version  of  the  light-field  display,  the  window  was  so  blurred  that  it  interfered  with  the  rod 
appearance  and  could  not  be  used. 

The  inability  of  the  FoLD  systems  (AIS,  ZIR)  to  maintain  contrast  as  the  bars  moved  in  depth 
limited  the  James  stimulus  presentation  to  white  bars  against  a  black  background  configuration. 
In  the  ZIR,  some  of  the  lost  contrast  is  due  to  hogel  lens  glow.  When  presenting  a  black  rod 
against  a  white  background  the  light  beam  bundle  (coming  from  pixels  dispersed  across  several 
hogels)  through  the  rod  position  to  the  eye  should  be  very  low.  However,  to  support  head 
motion  parallax,  that  specific  hogel  must  also  contributes  to  the  white  background  image  for  a 
very  large  number  of  alternative  viewing  positions.  Consequently  a  large  percentage  of  the  5776 
potential  beams  generated  by  each  hogel  are  illuminated.  Each  of  these  beams  produces  a  small 
reflection  off  every  optical  surface  it  crosses  in  the  hogel  micro-lens  system.  The  cumulative 
effect  of  the  very  large  number  of  small  reflections  is  that  the  hogel  lens  starts  to  glow,  washing 
out  the  desired  dark  appearance  of  the  rod. 

Arguably  the  most  interesting  activity  conducted  with  the  ZMD  in  this  effort  was  the  calibration 
in  depth.  The  ability  to  refocus  display  images  at  different  depths  confirms  that  the  ZMD  hogel 
array  is  producing  a  legitimate  lightfield  when  viewed  with  a  50mm  diameter  aperture.  In 
addition,  the  bivariate  data  is  compatible  with  a  wide  variety  of  analyses.  For  example,  a 
multiple  regression  analysis  could  determine  whether  linear  or  curvilinear  function  is  a  better  fit 
to  the  data.  In  the  data  shown  in  Figure  19,  adding  a  Command  Position  squared  term  to  the 
regression  equation  raises  R2to  0.963  with  the  squared  term  being  significant  at  P  <  0.001, 
suggesting  the  ZMD  depth  response  is  not  entirely  linear.  Another  potential  approach  for 
quantifying  the  quality  of  the  light  field  would  be  reduce  the  aperture  of  the  focusing  lens  until 
the  regression  curve  slope  is  no  longer  distinguishable  from  zero. 
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5.0  CONCLUSIONS  AND  RECOMMENDATIONS 


5.1  Conclusions 

The  report  documents  an  initial  effort  to  develop  a  quantitative  metric  for  the  perception  of  depth 
on  electronic  3D  displays  at  6m.  The  metric  is  based  on  a  test  and  physical  depth  perception 
apparatus  (DPA)  invented  by  James  (1907,  1908),  an  ophthalmologist  working  in  the  UK.  The 
James  test  and  physical  apparatus  were  subsequently  used  and  advanced  by  other  research 
ophthalmologists  in  The  Netherlands  (Schoiite  1910)  and  the  U.S.  (Howard  1919a,  1919b),  and 
by  a  research  psychologist  in  the  U.S.  (Deyo  1922).  The  6m  viewing  distance  was  chosen  to 
enable  correlation  of  depth  acuity  to  lateral  acuity  per  Snellen  (1862). 

The  James  test  (James  1908)  was  selected  as  the  basis  for  the  assessment  depth  perception  of 
computer-generated  (i.e.  virtual)  imagery  on  various  electronic  3D  displays.  The  foundations  for 
an  electronic  James  (eJ)  test  and  methodology  were  established. 

The  Howard  Six  Meter  Stereoscope  (Howard  1919b)  enabled  the  use  of  two  measurement 
approaches:  MoCS  and  MoA.  A  variant  of  this  apparatus  developed  at  71 1  HPW/USAFSAM 
was  automated  at  71 1  HPW/RHCV  for  use  as  the  reference  device  in  the  present  work.  This 
device  served  well  as  a  reference  in  the  present  work,  but  the  interior  lighting  scheme  (black  rods 
lit  from  front,  back  and  sides)  was  found  to  be  inferior  to  that  of  Howard  DPA  (black  rods  lit 
from  behind  only). 

Three  electronic  3D  displays  were  tested  and  compared  to  this  AFRL-modified  Howard  DPA. 
Two  types  of  full-parallax  prototype  systems  in  the  AFRL  711  HPW/RHCV  Visualization 
Laboratory  were  among  the  devices  under  test  (DUT):  the  Actuality  Integral  Slice  (AIS) 
Perspecta  spinning  screen  volumetric;  and  the  Zebra  hogel-based  Zspace  Integral  Ray  (ZIR).  For 
comparison,  one  type  of  stereo  3D  (S3D)  device  (an  active  eyewear  laptop)  was  included.  The 
AIS  and  ZIR  are  examples  of  two  types  within  a  display  class  (full  parallax)  that  are  herein 
called  Field  of  Light  Displays  (FoLD).  These  FoLD  systems  (AIS,  ZIR)  were  engineering 
prototypes,  not  commercial  products,  and  both  had  significant  visual  artifacts.  The  S3D  device 
also  had  image  rendering  limitations.  All  three  electronic  displays  require  engineering  and 
software  improvements  before  they  can  support  high  quality  depth  perception  measurements. 

The  choice  of  rod  color  (white),  background  color  (black),  and  lighting  direction  (front  side)  is 
the  same  as  used  by  James  (1907,  1908);  this  choice  was  forced  by  the  image  stimuli  rendering 
and  display  limitations  of  the  AIS,  ZIR,  and  S3D  devices.  And  the  viewing  window  used  in  all 
physical  DPA  devices  was  not  included  in  the  electronic  stimuli  for  the  same  reasons.  These 
engineering  artifacts  and  rendering  issues  (a)  impacted  the  design  of  electronically  generated 
images  that  were  perceptually  comparable  to  the  two-bar  stimuli  viewed  in  the  DPA  and  (b) 
significantly  degraded  stereoacuity  threshold  measurements  relative  to  the  physical  DPA. 
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During  this  project  it  was  determined  that  the  seminal  work  by  Howard  (1919a,  1919b,  1919c, 
1919d,  1920a,  1920b,  1920c)  was  performed  at  the  Medical  Research  Laboratory  (MRL), 
Hazelhurst  Field,  Minneola  NY  (established  in  1918),  which  is  the  initial  version  of  the  represent 
day  AFRL  Airmen  Systems  Directorate  (USAF  AFMC  711  HPW/RH,  WPAFB  OH).  And  the 
work  performed  by  Deyo  (1922)  and  Tefft  et  al.  (1922,  1923)  was  performed  at  MRL  and  the  co¬ 
located  School  for  Flight  Surgeons  (SFS),  the  initial  version  of  the  present  day  USAF  School  of 
Aerospace  Medicine  (711  HPW/USAFSAM).  That  is,  the  authors  of  this  report  are  the  heirs  of 
seminal  research  on  the  topic  of  depth  perception.  All  original  literature  from  the  work 
performed  from  1918-1925  at  Hazelhurst  Field  was  found  at  archival  library  files  now  located  at 
WPAFB  OH.  A  thorough  examination  of  this  literature  showed  that  all  literature  references  to  a 
so-called  “Howard-Dolman”  DPA  should  have  been  to  the  “Howard-Deyo”  apparatus. 

The  Howard-Deyo  DPA  (pictured  in  Figure  6)  is  a  hybrid  of  the  James  (1907,  1908)  DPA 
pictured  in  Figure  3b  and  the  Howard  (1919b)  Six  Meter  Stereoscope  illustrated  in  Figure  5.  The 
Howard-Deyo  DPA  was  not  used  in  the  present  work. 

The  developmental  effort,  described  in  this  report,  was  an  attempt  to  apply  a  very  simple 
psychophysical  measurement  to  a  variety  of  electronic  3D  display  devices  in  an  effort  to  start 
exploring  the  relationship  between  the  perception  of  depth  and  physical  characteristics  of  the 
displays.  Despite  the  simplicity  of  the  psychophysical  measurement  chosen,  the  developmental 
effort  immediately  ran  into  difficulty  due  to  the  unique  characteristics  and  limitations  of  each 
display  device.  No  common  stimulus  configuration  could  be  found  that  was  suitable  for  all 
display  types.  This  was  initially  disappointing  in  that  the  failure  to  create  similar  imagery  on 
each  display  limits  the  ability  to  conduct  cross  display  category  comparisons.  Ultimately, 
working  with  the  unique  characteristics  of  each  display  was  extremely  informative  in  regard  to 
their  current  level  of  maturity  and  their  theoretical  potential. 

Previously  3D  displays  have  either  been  thought  of  as  an  entertainment  device  where  the  depth 
dimension  is  either  (a)  something  to  be  manipulated  by  artist  for  content  creation  or  (b)  an 
information  device  assumed  to  be  accurate  and  helpful.  If  an  individual  had  difficulty  extracting 
high  quality  depth  information  from  the  display  it  was  often  assumed  to  be  a  problem  with  the 
observer,  not  a  limitation  of  the  device;  but,  as  the  understanding  of  3D  displays  improves,  this 
assumption  is  not  sustainable.  The  inevitable  expansion  of  3D  display  options  from  stereo  to 
include  holographic,  volumetric,  and  other  light-field  displays,  will  eventually  require  methods 
for  measuring  the  quality  of  a  3D  display  performance  on  their  defining  characteristic,  depth. 
Developing  these  methods  will  be  a  challenge,  as  depth,  in  at  least  some  of  these  displays,  will 
exist  only  as  a  perception  similar  to  brightness  and  color. 
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5.2  Recommendations 


A  virtual,  electronic  James  (eJ)  test  was  successfully  demonstrated  on  each  display  type  and  is 
available  for  use  in  with-in  category  display  comparisons. 

It  is  recommended  higher  quality  FoLD  devices  be  created  and  the  electronic  James  (eJ)  test  be 
further  developed  to  guide  research  and,  eventually,  quality  control  in  products. 

All  references  to  “Howard-Dolman”  should  be  corrected  to  read  “Howard-Deyo.” 

The  automation  of  subject  testing  in  this  effort  expedited  data  collection  and  should  be  used. 

Prior  work  with  manual  testing  and  data  recording  can  be  speed  up  dramatically  using  either  the 
MoCS  or  MoA  approaches.  Research  reported  after  the  1907-1922  period  has  avoided  frequent 
use  of  the  MoCS  approach  due  to  its  labor-intensive  nature  for  subject  testing;  with  automated 
testing  and  data  collection  the  MoCS  approach  is  even  faster  to  administer  than  the  MoA. 

In  the  past,  psychophysical  measurements  have  been  very  successfully  in  tying  perceptions  to 
physical  measures.  For  example,  the  perception  of  brightness  is  linked  to  the  physical  measure  of 
luminance,  and  color  is  linked  to  chromaticity.  A  similar  measure  of  depth  needs  to  be  matured 
for  electronic  3D  displays. 
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LIST  OF  SYMBOLS,  ABBREVIATIONS  AND  ACRONYMS 


2AFC 

2D 

3D 

9-tile 

x 

Xretina 

Z 

Actuality 

AFRL 

AIS 
A  JO 

AMOLED 

AOC 

AOI 

AOS 

API 

ASP 


AVE 

BP 

BPA 

BSV 

BSD 

CAD 

COTS 

CREST 

CT 

DARPA 

Dim 

DLP 

DMD 

DoD 

DPS 

DXT 

eJSS 

EMD 

FMHD 

FOB 


Two  Alternative  Forced  Choice 
Two  dimensional 
Three  dimensional 

Configured  ZMD  with  9  tileable  units  to  create  21”  diagonal  display 
Linear  lateral  displacement  in  the  image  plane  for  object  moved  from  that 
plane 

Linear  projection  on  the  retina  for  xx 

Linear  depth  displacement  of  line  of  sight  to  an  object  moved  from  that  plane 
Volumetric  3D  based  DMD  image  slices  projected  onto  spinning  screen 
Air  Force  Research  Laboratory 

Actuality  Integral  Slice  (“Perspecta”),  a  volumetric  FoLD  system 

American  Journal  of  Ophthalmology 

Active  Matrix  Organic  Light  Emitting  Diode 

Air  Force  Air  Operations  Center 

Area  of  interest 

American  Ophthalmology  Society 
Application  programming  interface 

Angle  of  Stereoscopic  Parallax  =  angle  subtended  at  eye  nodal  point  by  x=SP 
=  BPA  =  difference  between  apex  angles  subtended  at  two  objects  by  the  IPD 
baseline. 

Example:  for  (IPD,  z)  =  (63mm,  5mm),  ASP  =  8.74  prad  =  1.803  arcsec. 
Augmented  virtual  environment 

Binocular  Parallax  (same  as  Linear  Stereocopic  Parallax  =  linear  SP) 
Binocular  Parallactic  Angle  (term  used  by  Howard  1919a)  =  ASP 
Binocular  Single  Vision  =  the  perception  of  a  single  image  for  an  object  seen 
with  both  eyes 

Berkley  Software  Distribution 
Computer-aided  design 
Custom  off-the-shelf 

Center  for  Research  in  Education  and  Simulation  Technologies 

Computerized  tomography 

Defense  Agency  Research  Projects  Agency 

Dimension 

Digital  Light  Processing 
Digital  micromirror  device 
Department  of  Defense 
Display  updates  per  second 
Compression  algorithm 
eJ  Software  Suite 

High  Resolution  Affordable  Emissive  Micro  Display  (a  DARPA  program) 
Full  Multiplex  Holographic  Display  (an  AFRL  program) 

Forward  operating  base 
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Fold 

Field  of  Light  Display  (volumetric  integral  slice,  hogel-based  integral  ray. 

FOV 

etc.) 

Field  of  views 

FPGA 

Field  programmable  gate  array 

FPS 

Frames  per  second 

FPS 

Frames  per  second 

FRU 

Field  replaceable  unit 

GL/ES 

Graphical  language  /  Embedded  systems 

GPS 

Global  positioning  system 

GRIN 

Gradient  Index  Lens 

GUI 

Graphical  user  interface 

GWOT 

Global  War  on  Terror 

HL3DM 

Holographic  Lightfield  3D  Display  Metrology  (an  AFRL  program) 

HD 

High  definition 

HDDPA 

Howard-Devo  Depth  Perception  Apparatus 

Howard-Dolman  Depth  Perception  Apparatus  (Dolman  should  be  Devo) 

Hogel 

Holographic  Element 

HPO 

Horizontal  parallax  only 

HPS 

Hogels  per  second 

HVD 

Holographic  Video  Display  (an  AFRL  program) 

Hz 

Hertz 

LO 

Input/output 

ISSO 

Information  Services  Support  Office 

IVIR 

Information  Visualization  and  Innovative  Research 

LCOS 

Liquid  crystal  on  silicon 

LDAM 

Light  Distribution  Array  Modification 

LED 

Light  emitting  diode 

LiDAR 

Light  detection  and  ranging 

Linear  SP 

Same  as  SP 

Matlab 

Tool  for  doing  numerical  computations  with  matrices  and  vectors 

MEMS 

Micro-electro-mechanical  syste. 

mopel 

micro-optical  element  (aka  hogel) 

Mpx 

Megapixel 

MRI 

Magnetic  resonance  imaging 

MTF 

Modulation  transfer  function 

MVP 

Model  View  Projection  Matrix 

OGC 

Open  Geospatial  Consortium 

OIF 

Operation  Iraqi  Freedom 

OS 

Operating  system 

OTS 

Off-the-shelf 

PBM 

Performance  Based  Metrics 

PC 

Personal  computer 

PCB 

Printed  circuit  board 

PCI-e 

Peripheral  Component  Interconnect  Express 

PM/CM 

Project  Management  /  Contracts  Management 
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PMO 

POC 

POV 

PRISM 

PRS 


PSF 

RDECOM 

RGB  (+D) 

RMS 

S3D 

SAR 

SBIR 

SD 

SEM 

SHO 

SIR 

SLM 

SOC 

SOCOM 

SP 


SPDU 
Stereo  3D 


Project  Management  Office 

Proof-of-concept 

Point  of  view 

Parallel  Ray  Interpolation  for  Stereo  Mosaicing 

Actuality  Integral  Slice  (AIS)  Perspecta  Rotating-Screen  system  (a  FoLD 
system  developed  by  Actuality  Integral  Slice  (AIS)  Perspecta  . . .  that  creates  a 
temporally-averaged  perspective  3D  image  at  each  ~5mm  pupil  position  in 
the  design  360x1 20-deg  HxV  near-spherical  viewing  volume  via  rapid 
sequential  projection  of  circular  slices  of  the  3D  model  data  as  the  screen 
spins;  each  slice  must  be  refreshed  at  >30  Hz 
Point-spread  function 

Army  Research,  Development  and  Engineering  Command 
Red,  blue,  green,  plus  depth 
Root  mean  squared 
Stereo  3D  (display  device) 

Synthetic  aperture  ranging 

Small  Business  Innovative  Research  (a  government  program) 

Standard  Deviation  (  a  ),  defined  if  Gaussian  statistical  model  applies  (n>8) 
Standard  Error  Measure,  for  n  =  6  (Gaussian  statistics  not  applicable)-this 
report 

Synthetic  Holographic  Observation  (an  IARPA  program) 

Spatially  Integrated  Ray  (a  type  of  light  field 
Spatial  light  modulator  (modulation) 

Systems-on-a-chip 
Special  Operations  Command 

Stereoscopic  Parallax  =  the  linear  distance  (in  the  image  plane)  between  two 
lines  projected  from  one  eye  nodal  point  to  two  object  points  (one  in  and  one 
beyond  the  image  plane);  also  called  linear  SP,  or  Binocular  Parallax 
The  two  object  points  are  both  on  line  projected  from  the  other  eye  nodal 
point. 

The  image  plane  is  parallel  to  the  IPD  line  between  the  two  nodal  points. 
Second  per  display  update 

Left/right  eye  images  projected  sequentially  viewed  with  special  viewing 
glasses  (eye  selection  wear,  active  shutter  or  left/right  circularly  polarized) 


svo 

SWaP 

TAOS 

TBV 

TCPIP 

TDH 

TI 

UPSD 

USSOCOM 

YE 


Sparse  voxel  octree 
Size,  weight,  and  power 

Transactions  of  the  Americal  Ophthalmological  Society 

Tactical  Battlefield  Visualization 

Transmission  Control  Protocol/Internet  Protocol 

Tactical  Digital  Holograms 

Texas  Instruments 

Urban  Photonic  Sandtable  Display 

United  States  Special  Operations  Command 

Virtual  environment 
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Zebra  Integral -Ray  system  (a  full-parallax  FoLD  system  developed  by  Zebra 
Imaging,  Inc.  that  creates  an  image  at  each  ~5mm  pupil  position  in  the  design  90- 
deg.  conical  viewing  zone  as  from  rays  emanating  from  widely  distributed,  non- 
adjacent  2D  pixels,  each  projected  in  a  fixed  direction  by  a  microoptics  array) 
ZScape®  Motion  Display 
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ZIR 


ZMD 


